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FOREWORD 


The  U.S.  Army  has  made  a  substantial  commitment  to 
Distributed  Interactive  Simulation  (DIS)  and  the  electronic 
battlefield  for  training,  concept  development,  and  test  and 
evaluation.  The  current  DIS  training  system.  Simulation  Net¬ 
working  (SIMNET) ,  and  the  next  generation  system,  the  Close 
Combat  Tactical  Trainer  (CCTT) ,  provide  effective  training  for 
soldiers  fighting  from  vehicles,  but  are  unable  to  do  the  same 
for  individual  dismounted  soldiers.  Virtual  Environment  (VE) 
technology  has  the  potential  to  provide  Individual  Combat  Simu¬ 
lations  (ICS)  for  the  electronic  battlefield.  However,  several 
research  challenges  must  be  overcome  before  VE  technology  can  be 
used  for  practical  training  applications.  These  challenges 
include:  (a)  providing  all  trainees  with  the  necessary  prereq¬ 
uisite  skills  for  operating  in  VEs;  (b)  identifying  and  quanti¬ 
fying  the  effects  of  VE  system  characteristics  that  influence 
skill  acquisition  and  transfer;  and  (c)  ensuring  that  unwanted 
side  effects  and  aftereffects  that  might  result  from  immersion  in 
VEs  do  not  pose  unacceptable  risks. 

This  report  describes  our  initial  research  conducted  to 
obtain  quantitative  data  on  human  performance  and  learning  in 
VEs,  and  the  research  tools  developed  for  this  and  future 
research.  Portions  of  the  research  have  been  presented  at  a 
variety  of  Army,  Department  of  Defense,  and  scientific  confer¬ 
ences.  A  less  detailed  version  of  this  report  was  published  in 
Presence:  Teleoperators  and  Virtual  Environments,  Volume  3, 

Number  2,  Spring  1994. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  Simulator  Systems  Research  Unit  conducts  research 
to  improve  the  effectiveness  of  training  simulators  and  simula¬ 
tions.  The  work  described  is  a  part  of  the  ARI  research  task 
titled  VIRTUE— Virtual  Environments  for  Combat  Training  and 
Mission  Rehearsal. 
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Director 


THE  VIRTUAL  ENVIRONMENT  PERFORMANCE  ASSESSMENT  BATTERY  (VEPAB) : 
DEVELOPMENT  AND  EVALUATION 


EXECUTIVE  SUMMARY 


Requirement : 

The  U.S.  Army  has  made  a  substantial  commitment  to  the  use 
of  virtual  environment  (VE)  technology,  such  as  networked  simula¬ 
tors  to  create  virtual  battlefields  for  combat  training  and 
mission  rehearsal,  development  of  military  doctrine,  and  evalua¬ 
tion  of  weapon  system  concepts  prior  to  acquisition  decisions. 

All  of  these  functions  would  be  improved  by  a  better  representa¬ 
tion  of  dismounted  infantry  on  the  virtual  battlefield.  Immer¬ 
sive  VE  technology,  also  known  as  virtual  reality,  may  provide  an 
interface  to  allow  dismounted  soldiers  to  fight  on  virtual 
battlefields.  However,  several  research  challenges  must  be  met 
before  this  technology  can  be  applied.  These  challenges  include 
(a)  providing  all  trainees  with  the  necessary  prerequisite  skills 
for  operating  in  VEs;  (b)  identifying  and  quantifying  the  effects 
of  VE  system  characteristics  that  influence  skill  acquisition  and 
transfer;  and  (c)  ensuring  that  unwanted  side  effects  and  after¬ 
effects  that  might  result  from  immersion  in  VEs  do  not  pose 
unacceptable  risks. 


Procedure : 

The  Virtual  Environment  Performance  Assessment  Battery 
(VEPAB),  a  set  of  VE  perceptual  and  psychomotor  tasks,  was 
developed  to  support  research  on  training  applications  of  VE 
technology.  Tasks  were  developed  for  each  of  five  categories: 
vision,  locomotion,  tracking,  object  manipulation,  and  reaction 
time.  These  tasks  are  components  of  essential  soldier  functions 
(move,  communicate,  and  employ  weapons) .  The  tasks  were  adminis¬ 
tered  to  college  students  and  government  employees  to  measure  the 
initial  level  of  performance,  practice  effects,  the  effects  of 
different  interface  devices,  and  the  incidence  and  severity  of 
side  effects  and  aftereffects  resulting  from  VE  immersion. 


Findings: 

Performance  on  most  VEPAB  tasks  was  sensitive  to  differences 
between  input  control  devices.  Most  were  also  sensitive  to  prac¬ 
tice  effects.  All  of  the  tasks  could  be  performed  by  our  diverse 
group  of  participants.  However,  significant  variation  was 
observed  across  individuals  in  their  initial  abilities  to  perform 


vii 


tasks  in  VEs.  Individual  differences  were  also  observed  in  self 
reports  of  the  occurrence  and  severity  of  symptoms  of  discomfort 
resulting  from  VE  immersion.  Most  individuals  enjoyed  the  VE 
experience.  However,  most  reported  some  symptoms  and  one  of  24 
withdrew  because  of  symptoms  similar  to  those  of  motion  sickness 


Utilization  of  Findings: 

The  results  of  this  research  confirm  that  the  VEPAB  can  be 
an  effective  research  tool.  Overall,  task  performance  is  sensi¬ 
tive  to  the  effects  of  different  control  devices  and  the  amount 
of  practice  performing  the  tasks.  It  is  also  sufficiently 
reliable.  The  tasks  will  be  used  in  future  experiments  investi¬ 
gating  the  effects  of  different  visual  display  devices  on  task 
performance  and  training. 
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THE  VIRTUAL  ENVIRONMENT  PERFORMANCE  ASSESSMENT  BATTERY  (VEPAB) : 

DEVELOPMENT  AND  EVALUATION 

Introduction 

This  report  describes  the  development  and  initial  evaluation 
of  an  integrated  battery  of  tasks  to  measure  human  performance  in 
immersive  virtual  environments  (VEs) .  Immersive  VEs,  sometimes 
referred  to  as  Virtual  Reality,  have  been  defined  as  human 
computer  interfaces  "in  which  the  computer  creates  a  sensory- 
immersing  environment  that  interactively  responds  to  and  is 
controlled  by  the  behavior  of  the  user"  (Pimentel  and  Teixeira, 
1993,  p.  15).  At  the  time  we  designed  VEPAB  there  was  little  or 
no  published  information  on  how  people  perform  in  immersive  VEs. 
The  battery  was  developed  to  support  a  research  program  on 
training  applications  of  immersive  VE  technologies,  A  goal  of 
the  program  is  to  determine  those  technologies  that  produce  cost- 
effective  transfer  of  training  from  VE  practice  to  real-world 
performance,  and  strategies  for  the  effective  use  of  these 
technologies . 

The  U.S.  Army  has  made  a  substantial  commitment  to  the  use 
of  networked  simulators  to  create  virtual  battlefields  for  combat 
training  (Alluisi,  1991;  Sterling,  1993) .  VE  training  systems  in 
use,  such  as  Simulator  Networking  (SIMNET) ,  and  under 
development,  such  as  the  Close  Combat  Tactical  Trainer  (CCTT) , 
provide  training  for  soldiers  fighting  from  within  vehicles  such 
as  tanks  and  helicopters.  In  these  training  systems,  crew 
members  operate  inside  physical  mockups  of  combat  vehicles  with 
video  monitors  providing  their  views  of  the  simulated  outside 
world  (the  virtual  battlefield)  as  seen  through  vision  blocks  of 
armored  vehicles,  cockpits  of  aircraft,  and  the  electro-optical 
sights  of  weapon  systems.  Trainees  crouch,  sit,  or  recline  in 
the  simulators  in  positions  similar  to  those  they  would  assume  in 
actual  combat  vehicles.  The  ways  in  which  the  crew  members  view 
and  hear  the  world,  control  vehicle  movement,  and  employ  weapon 
systems  correspond  closely  to  those  of  actual  combat  vehicles. 

In  marked  contrast,  the  representation  of  dismounted  infantry  is 
limited  to  a  unit  leader,  seated  at  a  workstation,  controlling  a 
group  of  icons  that  represent  dismounted  soldiers.  This  approach 
may  train  mounted  soldiers  to  fight  with  and  against  dismounted 
soldiers,  but  it  is  inadequate  for  training  the  dismounted 
soldiers  themselves . 

Gorman  (1990)  proposed  that  immersive  VE  technologies  could 
provide  an  interface  to  enable  dismounted  soldiers  to  train  on 
virtual  battlefields.  With  this  approach  a  head-mounted  display 
(HMD)  would  present  a  view  of  a  computer-generated,  three- 
dimensional  environment  relative  to  an  eye  point  within  the 
environment.  The  human  user  could  control  the  direction  and 
movement  of  the  eye  point  and  interact  with  simulated  objects 
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within  the  VE.  In  this  manner  the  user  is  "immersed"  in  the  VE. 
This  interface  would  allow  trainees  to  move,  shoot,  and 
communicate  in  VEs.  Simulation  of  these  three  basic  soldier 
functions  could  allow  training  on  complex  scenarios  requiring 
real-time  tactical  decision  making. 

Gorman  (1990)  has  also  emphasized  the  importance  of  enabling 
dismounted  troops  to  train  on  virtual  battlefields.  In  addition 
to  training  applications,  virtual  battlefields  can  support 
several  other  functions  important  to  the  U.S.  military  (Alluisi, 
1991) .  For  example,  virtual  battlefields  can  be  used  to  develop 
and  evaluate  military  doctrine  and  to  evaluate  weapon  system 
concepts  prior  to  acquisition  decisions.  All  of  these  functions 
would  be  improved  by  a  better  representation  of  dismounted 
infantry . 

The  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  has  established  a  research  program  to  determine 
the  characteristics  of  the  VE  technologies  needed  to  provide  an 
interface  for  dismounted  soldiers  to  train  with  virtual 
battlefields  such  as  SIMNET  and  CCTT.  As  part  of  this  effort, 

ARI  contracted  with  the  University  of  Central  Florida's  Institute 
for  Simulation  and  Training  (1ST)  to  develop  a  laboratory  for  the 
conduct  of  psychological  research  addressing  human  performance  in 
VEs.  1ST  provides  computer  science  expertise  in  the  development 
and  operation  of  the  laboratory,  acquiring  off-the-shelf  system 
components  such  as  hardware  and  software  for  the  laboratory,  or 
developing  them  when  necessary.  1ST  scientists  predicted  rapid 
improvements  in  capabilities  and  prices  of  VE  technology, 
therefore  ARI  decided  to  begin  with  equipment  sufficient  to 
support  initial  research  (a  low-cost  stereoscopic  HMD,  a  head 
tracker,  a  manual  controller  device)  and  to  expand  the  testbed  as 
necessary.  Development  of  the  testbed  is  described  in  greater 
detail  by  Moshell,  Blau,  Knerr,  Lampton,  and  Bliss  (1993) . 

Currently  available  immersive  VE  technologies  have  high 
costs  and  performance  limitations,  such  as  the  limited  resolution 
of  HMDs,  which  preclude  the  immediate,  widespread  application  of 
those  technologies  for  training  dismounted  soldiers  (Levison  & 
Pew,  1993) .  A  long  term  goal  of  our  research  is  to  demonstrate 
that  immersive  VE  training  enables  the  learning  and  practice  of 
skills  that  transfer  to  field  training  and  can  be  e3<pected  to 
transfer  to  actual  combat. 

We  believe  that  our  research  with  the  admittedly  limited 
immersive  VE  technology  available  today  can  guide  the  timely  and 
cost-effective  development  of  VE  training  systems  as  technology 
improves.  In  networked  training  simulations  there  will  be  limits 
on  the  kinds  and  amount  of  information  that  can  be  passed  across 
the  network.  In  addition,  the  requirement  to  network  many 
trainees  simultaneously  will  place  an  emphasis  of  using 
inexpensive  equipment. 
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Pimentel  and  Teixeira  (1993)  have  pointed  out  that  absolute 
realism  may  not  be  necessary  to  create  a  sense  of  immersion;  the 
created  world  need  only  be  real  enough  for  the  user  to  suspend 
disbelief  for  a  period  of  time.  They  compared  immersion  in 
virtual  reality  to  being  absorbed  in  a  good  novel  or  a  computer 
game.  Pimentel  and  Teixeira  stated  that  interactivity  has  two 
primary  components;  navigation  and  the  dynamics  of  the 
environment.  Navigation  is  the  user's  ability  to  move  about  in 
the  environment .  The  dynamics  of  the  environment  are  determined 
by  the  rules  for  how  the  contents,  people  or  manipulable  objects 
for  example,  can  interact. 

Initial  Research  Considerations 


Several  research  challenges  must  be  overcome  before  VE 
technology  can  be  used  for  practical  training  applications. 

These  challenges  include:  (a)  providing  all  trainees  with  the 
necessary  prerequisite  skills  for  operating  in  VEs;  (b) 
identifying  and  quantifying  the  effects  of  VE  system 
characteristics  that  influence  skill  acquisition  and  transfer; 
and  (c)  insuring  that  unwanted  side  effects  and  aftereffects 
which  might  result  from  immersion  in  VEs  do  not  pose  unacceptable 
risks . 

Individual  Differences  in  VE  Performance 


In  order  to  engage  in  meaningful  training,  trainees  must 
have  at  least  some  minimum  level  of  skill  in  perceiving,  moving 
through,  and  manipulating  objects  in  the  VE.  For  example,  we 
created  a  prototype  training  scenario  that  involved  some  of  the 
action  which  might  be  performed  in  a  hostage  rescue  mission.  The 
scenario  required  individuals  to  explore  and  search  a  VE  model  of 
an  art  museum.  There  were  substantial  individual  differences  in 
initial  skills  in  operating  in  a  VE.  Some  individuals  could  not 
find  or  pass  through  the  front  door  of  the  building,  much  less 
learn  the  interior  configuration  of  the  building.  This  tended  to 
confirm  our  informal  observations  made  at  a  variety  of  VE 
demonstrations  that  for  many  individuals  task  performance  in 
those  VE  implementations  is  not  "intuitive."  Identification  of 
the  sources  of  the  individual  differences  may  be  an  interesting 
research  topic;  however,  for  the  short  term  an  important  need  is 
to  provide  a  systematic  familiarization  to  VEs,  that  is,  a  method 
to  train  basic  VE  skills.  In  addition,  a  method  is  needed  to 
measure  those  skills  in  order  to  determine  that  individuals  have 
achieved  at  least  some  minimiim  level  of  VE  skill  before 
proceeding  with  training  on  more  complex  tasks.  These  measures 
can  also  be  used  as  covariates  to  strengthen  analyses  of 
performance  on  complex  tasks  performed  in  VEs. 
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Measuring  Human  Performance  as  a  Function  of  VE  System 
Characteristics 


Development  of  effective  VE  training  systems  will  require 
many  tradeoff  decisions  in  the  selection  of  components  and  system 
operating  parameters.  Examples  of  these  decisions  are:  What  is 
an  appropriate  tradeoff  between  width  of  the  field  of  view  versus 
the  resolution  of  the  visual  display?  What  is  an  appropriate 
balance  of  scene  detail  versus  frame  rate?  Empirical  data  to 
support  these  decisions  are  limited.  There  is  some  relevant 
previous  research.  For  example,  Padmos  and  Milders  (1992) 
present  a  comprehensive  list  of  quality  criteria  for  visual 
displays  in  vehicle  simulators  used  for  training.  The  list 
includes  spatial  resolution,  contrast  ratio,  chromatic  color 
attributes,  field  size,  luminance,  update  frequency,  and  refresh 
rate.  However,  for  new  technologies  such  as  immersive  VE  the 
adequacy  of  the  visual  presentation  to  support  training  can  not 
be  predicted  solely  from  specification  of  the  physical 
characteristics  of  individual  components,  e.g.,  pixel  density  of 
the  HMD,  due  to  the  complex  interactions  of  the  many  components 
that  make  up  a  VE  system.  Furthermore,  subjective  judgements  of 
what  "looks  best"  may  not  accurately  predict  task  performance. 
Empirical  measures  of  hviman  performance  are  needed  to  allow 
comparisons  of  different  VE  components  and  system  parameters . 

Concerns  about  Unwanted  Side  Effects  and  Aftereffects 

There  are  anecdotal  reports  that  immersive  VEs  can  lead  to 
symptoms  similar  to  motion  sickness  symptoms.  Kennedy,  Lane, 
Lilienthal,  Berbaum,  &  Hettinger  (1992)  documented  the  frequency 
and  severity  of  such  symptoms  associated  with  training  in  flight 
simulators,  but  similar  data  for  immersive  systems  are  lacking. 
For  practical  applications,  even  mild  effects  may  diminish 
training  effectiveness. 

If  immersion  in  VEs  results  in  sickness  similar  to  that  which 
has  been  found  with  flight- training  simulators  then  the 
effectiveness  of  dismounted  infantry  training  using  HMDs  will  be 
diminished  in  several  ways.  For  example,  some  trainees  will  be 
unable  to  use  the  training  system,  others  will  be  able  to 
tolerate  sickness  but  training  will  be  diminished  through 
distraction  or  adoption  of  coping  strategies.  In  addition, 
aftereffects  will  pose  safety  problems.  In  the  short  term,  the 
most  important  consideration  is  that  we  need  a  realistic 
assessment  of  the  risks  to  the  participants  of  our  experiments. 

Development  of  the  Virtual  Environment  Performance  Assessment 

Battery  (VEPAB) 

We  developed  the  Virtual  Environment  Performance  Assessment 
Battery  (VEPAB)  to  provide  a  set  of  standard  materials  and 
procedures  to  investigate  the  issues  discussed  in  the  previous 
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section.  The  materials  are  simple  VEs  in  which  movement, 
tracking,  object  manipulation,  and  reaction  time  tasks  can  be 
performed.  The  procedures  are  the  instructions,  presented  to  VE 
users,  which  describe  the  tasks  to  be  performed  and  the  measures 
of  task  performance.  An  example  of  a  VEPAB  task  is  to  use  an 
input  control  device  to  "walk"  through  a  series  of  connected 
rooms .  The  instructions  direct  the  participants  to  move  through 
the  connecting  corridors  as  rapidly  as  possible  without  "bumping 
into"  walls  or  doorways.  Performance  measures  are  speed  (the 
time  to  move  through  each  successive  room)  and  accuracy  (the 
number  of  collisions  with  walls  or  door  frames) .  The  VEPAB  will 
allow  us  to  maintain  a  common  baseline  in  our  research  program  as 
new  VE  technologies  become  available,  different  training 
applications  are  investigated,  and  diverse  participant  pools  are 
employed. 

The  VEPAB  consists  of  a  set  of  simple  generic  tasks  that 
represent  components  of  more  complex  activities.  Simple  tasks, 
in  contrast  to  detailed  training  scenarios,  are  easier  to 
develop,  provide  a  general  context  applicable  to  other  areas  of 
training  research,  allow  isolation  of  critical  variables,  and 
facilitate  measurement  of  performance. 


Move 


SOLDIER  TASKS  VEPAB  FUNCTIONS 

Locomotion 

Vision 

Manipulation 
Reaction  Time 
(Audition) 
Tracking 

Figure  1 .  Relationships  between  VEPAB  tasks  and  soldier 
functions.  (Audition  Tasks  are  not  yet  developed.) 
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We  developed  tasks  for  each  of  five  categories:  vision, 
locomotion,  tracking,  object  manipulation,  and  reaction  time. 
These  tasks  are  components  of  the  soldier  tasks  in  which  we  are 
interested.  They  are  components  of  many  non-military  activities 
as  well.  Figure  1  depicts  the  relationship  of  the  VEPAB  tasks  to 
soldier  functions.  Although  the  VEPAB  tasks  are  clearly  related 
to  soldier  functions  the  tasks  were  designed  so  that  previous 
military  training  is  not  required  for  successful  task 
performance.  Therefore  civilians,  such  as  college  students,  can 
be  used  as  research  participants  in  testing  the  VEPAB  tasks . 

Table  1  provides  a  summary  of  VEPAB  tasks  by  task  category. 
We  developed  more  than  one  task  for  each  category  anticipating 
that  some  tasks  would  fail  to  meet  the  criteria  we  had 
established  and  therefore  be  unsuitable  for  future  use.  These 
criteria  included  adequate  reliability  and  an  appropriate  degree 
of  difficulty  (neither  too  difficult  nor  easy) .  Task  performance 
should  demonstrate  sensitivity  to  differences  in  display  and 
input  systems  and  show  improvement  with  practice.  Also,  we 
wished  to  have  available  a  pool  of  tasks  from  which  to  select, 
based  on  specific  research  requirements. 

In  order  to  implement  the  tasks  quickly  and  inexpensively, 
we  simplified  several  aspects  of  the  tasks  and  the  interface  with 
them.  Only  visual  cues  were  provided.  No  auditory  or  haptic 
devices  were  used.  The  participant's  body  was  not  visually 
represented  in  the  VE.  The  manipulation  tasks  were  performed 
using  the  same  control  devices  used  for  the  other  tasks.  The  use 
of  more  sophisticated  display  and  control  devices  (such  as  three- 
dimensional  sound,  instrumented  gloves,  and  instrumented 
treadmills) ,  and  visual  representation  of  the  human  figure  are 
important  areas  for  future  research.  However,  their 
investigation  was  beyond  the  scope  of  our  initial  effort. 

Both  because  of  the  importance  of  military  missions  in  urban 
areas,  such  as  hostage  rescue  or  house-to-house  searches  for 
weapons,  and  because  it  was  somewhat  easier  to  develop  realistic, 
visually  rich  building  interiors  than  exterior  terrain,  many  of 
the  VEPAB  tasks  are  situated  in  the  interiors  of  buildings. 
However,  the  interiors  have  no  explicit  military  connotations  and 
were  roughly  modeled  after  the  offices  which  we  occupied  when  we 
developed  the  VE  design  specifications.  The  level  of  detail  of 
the  interiors  was  selected  to  provide  a  variety  of  visual  cues  at 
an  acceptable  frame  update  rate.  Figure  2  illustrates  the  scheme 
used  in  modeling  the  building  interiors  for  the  locomotion  tasks : 
floors  have  checkerbbard  patterns,  walls  are  12  ft  high  with 
narrow  vertical  stripes  every  5  ft,  ceilings  have  horizontal 
light  panels  every  10  ft,  and  corridors  are  3  ft  wide. 
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Table  1 


Virtual  Environment  Performance  Assessment  Battery  Task 

Descriptions  by  Category 


TASK 

CATEGORY 


TASK  NAME 


TASK  DESCRIPTION 


Vision 


Acuity 


Color 


Object 

Recognition 


Size 

Estimation 


Distance 

Estimation 


Search 


Read  letters  in  a  Snellen  eye  chart 


Detect  colors  in  Ishihara  plates 


Identify  an  object  (a  human  figure) 
at  the  end  of  a  40  ft  hallway 


Estimate  the  height  of  a  human 
figure  at  the  end  of  a  40  ft  hallway 


Indicate  when  the  image  of  a  human 
figure,  moving  toward  the  viewer  from 
an  initial  distance  of  40  feet,  is  30, 
20,  10,  5,  and  2.5  ft  away 


Detect  a  target  moving  about  the 
walls,  floor,  or  ceiling  of  a  room 


Loco¬ 

motion 

(walking) 


Loco¬ 

motion 

(flying) 


Straight¬ 

away 


Back-up 


Turns 


Figure- 8 


Doorways 


Windows 


Elevator 


Move  down  a  straight  corridor  to  a 
circle  on  the  floor,  turn  around,  and 
return  to  the  starting  point 


Move  down  a  straight  corridor  to  a 
circle  on  the  floor,  then  move  back¬ 
wards  to  the  starting  point 


Move  through  a  corridor  formed  by  10 
alternating  left  and  right  90  degree 
turns 


Move  around  a  figure- 8  shaped 
corridor 


Move  through  a  series  of  rooms  con¬ 
nected  by  doorways  that  are  offset  so 
that  a  curved  course  must  be  followed 


Like  Doorways,  except  that  some  of  the 
openings  are  elevated,  so  that  verti¬ 
cal,  as  well  as  horizontal,  movement 
is  required 


Move  forward  through  a  structure  while 
going  over  or  under  a  series  of  verti¬ 
cal  obstacles 


Table  1  (continued) 


TASK 

CATEGORY 

TASK  NAME 

TASK  DESCRIPTION 

Tracking 

Head 

Control , 

Stationary- 

Target 

Use  head  movements  to  move  a  cursor, 
centered  in  the  viewing  device,  onto  a 
stationary  target 

Head 

Control , 
Moving 

Target 

Use  head  movements  to  move  a  cursor, 
centered  in  the  viewing  device,  onto  a 
target  moving  in  a  straight  line 

Device 
Control , 
Stationary 
Target 

Use  a  control  device  to  move  a  cursor 
onto  a  stationary  target 

Device 
Control , 
Moving 

Target 

Use  a  control  device  to  move  a  cursor 
onto  a  target  moving  in  a  straight 
line 

Manip¬ 

ulation 

Slide 

"Grasp"  a  control  bar  and  move  it  hor¬ 
izontally  to  a  marked  location 

Dial 

"Grasp"  a  dial  and  rotate  it  to  an 
indicated  orientation 

Bins 

"Grasp"  a  ball  located  in  a  vertical 
rack  of  open  containers  (bins) ,  pull 
it  out  of  the  original  bin,  and  push 
it  into  a  target  bin 

Reaction 

Time 

Simple 

Indicate  when  an  "X"  pops  into  view 

Choice 

Indicate  in  which  of  four  boxes  an  "X" 
has  appeared 
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Figure  2.  An  example  of  Interior  design:  The  intersection  of 
the  Pigure-8  task. 


VEPAB  Task  Descriptions 
Vision  Tasks 


The  vision  tasks  include  acuity,  color,  and  search.  Other 
vision  tasks  address  the  recognition  of  a  familiar  object,  a 
human  figure,  and  the  estimation  of  the  size  of,  and  distance  to, 
the  object.  These  visual  skills  are  important  to  the  performance 
of  many  real-world  tasks.  The  vision  tasks  require  participants 
to  report  aloud  what  they  perceive. 

Acuity 

A  Snellen  eye  chart,  a  virtual  reproduction  of  the  familiar 
letter  chart  found  in  every  doctor's  office,  was  created  by 
modeling  each  individual  letter.  The  chart  is  presented  at  eye 
level  at  the  end  of  a  20-foot  corridor.  The  participant  is 
instructed  to  describe  the  visual  scene  and  is  not  told  that 
there  is  an  eye  chart  at  the  end  of  the  corridor.  From  twenty 
feet  away  the  correct  identification  of  only  the  top  letter  of 
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the  chart  corresponds  to  an  acuity  of  20/200.  We  anticipated 
that  the  resolution  of  some  VE  visual  displays  would  result  in 
acuity  worse  than  20/200.  To  enable  the  measurement  of  acuity 
worse  than  20/200  the  VEPAB  acuity  task  allows  the  eye  point  to 
be  moved  toward  the  chart.  The  experimenter  moves  the 
participant's  eye  point  forward,  toward  the  chart,  in  one- foot 
intervals  and  records  the  distance  at  which  the  participant  first 
reads  the  top  line  of  the  chart.  Additional  measures  are  the 
nximber  of  lines  of  the  chart  that  can  be  read  at  each  distance 
interval . 

Color 


Standard  color  vision  test  plates  were  digitized  to  produce 
VE  tests  for  mild  (red-green)  and  severe  (blue-yellow)  color 
vision  deficiencies.  Three  circles  made  up  of  colored  dots 
appear  on  each  plate.  Dot  patterns  within  the  circles  form 
numerals.  Participants  read  the  numerals  aloud.  The  performance 
measure  is  the  number  of  numerals  correctly  recognized  for  each 
plate.  The  participant's  real-world  color  vision  is  measured 
before  this  VE  test  is  administered.  Thus,  this  task  is  a  gross 
measure  of  the  capability  of  the  VE  system  to  present  appropriate 
colors . 

Object  recognition,  size  estimation,  and  distance  estimation 

A  digitized  picture  of  a  human  figure  appears  at  the  end  of 
a  forty- foot  corridor.  The  participant  is  asked  to  identify  the 
object  and  estimate  its  height.  The  participant  is  told  the 
correct  height,  6  ft,  and  asked  to  estimate  the  distance  to  the 
figure.  The  experimenter  then  tells  the  participant  that  the 
figure  is  40  ft  away  and  that  the  figure  will  begin  to  move 
forward.  The  participant  calls  out  when  the  figure  appears  to  be 
30,  20,  10,  5,  and  2.5  ft  ("arms  length")  away. 

Search 


The  participant's  viewpoint  is  at  eye- level  in  the  center  of 
a  room.  A  red  ball  appears  near  the  floor,  ceiling,  or  walls, 
and  moves  slowly  around  the  room.  The  participant  searches  for 
the  target  by  making  head  movements  and  turning  in  their  chair, 
then  calls  out  when  the  target  is  detected.  One  practice  trial 
and  ten  performance  trials  are  conducted.  For  the  practice  trial 
the  target  appears  in  the  participant's  field  of  view.  For  the 
performance  trials,  the  starting  location  of  the  target  is 
determined  by  random  number  generation  with  the  restriction  that 
the  target  is  not  initially  in  the  participant's  field  of  view. 
The  performance  measure  is  the  time  to  report  the  target.  The 
Search  task  provides  a  transition  from  the  vision  tasks  to  motor 
tasks  in  that  the  Search  task  is  presented  more  than  once  and  a 
more  active  role  is  required  of  the  participant. 
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Locomotion  Tasks  (Walking) 


Many  VE  applications  will  require  participants  to  move  at 
realistic  rates  while  simultaneously  attending  to  other  tasks; 
the  cognitive  load  imposed  by  walking  in  VEs  should  not  be 
significantly  greater  than  real-world  walking.  We  designed  a 
series  of  progressively  more  complex  locomotion  tasks  to  system¬ 
atically  train  participants  to  move  at  a  reasonable  speed  while 
avoiding  collisions  with  obstacles  such  as  walls  and  door  frames. 
In  addition  to  providing  an  efficient  way  to  train  VE  locomotion, 
the  tasks  provide  objective  performance  measures,  and  indirectly 
provide  diagnostics  of  problem  areas. 

The  locomotion  tasks  require  the  participant  to  "walk" 
through  the  VE  by  using  an  input  control  device  to  direct  the 
speed  and  direction  of  the  participant's  simulated  body.  (We 
refer  to  this  mode  of  virtual  locomotion  as  walking  inasmuch  as 
the  height  and  rate  of  movement  of  the  viewpoint  are  constrained 
to  match  that  of  walking,  and  the  contexts  (hallways,  doorways, 
rooms)  are  those  in  which  walking  is  normally  conducted.)  The 
dimensions  of  the  virtual  body  represent  the  50th  percentile 
male;  the  height  is  68  in.,  elbow- to -elbow  breadth  is  16.5  in. 
(McCormick  &  Sanders,  1976) .  Eye  level  is  set  at  65  in.  above 
the  floor.  There  is  no  visible  representation  of  the  body.  The 
body  can  move  forward  or  backward,  laterally,  or  rotate.  Soft¬ 
ware  settings,  chosen  to  represent  normal  walking  parameters, 
control  the  maximum  speed  and  the  rates  of  acceleration  and 
rotation.  The  body  interacts  with  the  VE  through  collisions  with 
walls  and  door  frames.  Because  collisions  almost  always  stop 
forward  progress,  requiring  the  participant  to  back  up,  movement 
at  a  reasonable  rate  requires  emphasis  on  both  speed  and  accuracy 
of  movement. 

The  performance  measures  for  each  of  the  tasks  are  the  time 
to  complete  the  task  and  accuracy.  Accuracy  is  defined  as  the 
ntimber  of  collisions  with  walls  or  door  frames.  These  measures 
can  be  obtained  for  each  task  as  a  whole  and  the  individual 
segments,  e.g.,  the  turns  or  rooms,  that  make  up  the  task. 

Straightaway  (Figure  3) 

The  first  locomotion  task  requires  the  participant  to  move 
down  a  straight  corridor  to  a  target  location  indicated  by  a 
circle  on  the  floor,  turn  180  degrees,  and  return  to  the  starting 
point  which  is  also  indicated  by  a  circle  on  the  floor.  The 
walls  at  opposite  ends  of  the  VE  corridor  have  color  and  form 
cues  to  help  the  participant  orient. 

Backup 

This  task  is  conducted  in  the  same  VE  as  the  Straightaway 
task.  The  participant  moves  down  a  straight  corridor  to  the 
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RED  WALL 
GREEN  WALL 

Figure  3.  Overhead  view  of  the  Straightaway  and  Backup  tasks. 

target  location,  then  moves  backwards  to  the  starting  point 
without  turning  around. 

Both  the  Straightaway  and  Backup  tasks  can  be  conducted  as 
discrete  trials  or  as  continuous  movement.  For  discrete  trials 
the  experimenter  can  terminate  a  trial  when  the  participant 
announces  the  return  to  the  initial  starting  point.  For  continu¬ 
ous  movement,  the  participant  is  instructed  to  move  from  circle 
to  circle  until  told  to  stop. 

Turns  (Figure  4) 

The  task  consists  of  a  continuous  narrow  corridor  formed  by 
straightaways  joining  alternating  90  degree  turns  to  the  left  and 
right  for  a  total  of  ten  turns.  Lengths  of  the  straightaways  are 
varied,  with  two  twenty- foot  segments  alternating  with  two  ten- 
foot  segments. 

Figure- 8 

Two  adjoining  oval  corridors  form  a  Figure- 8  course.  A 
small  diameter  oval,  rounded  on  each  end  with  straightaways  in 
the  middle,  is  connected  to  an  oval  of  larger  diameter  to  form  a 
closed  loop.  Walking  through  the  course  without  colliding  with 
the  walls  requires  gradual  turns  to  the  left  and  right. 
Performance  measures  are  the  number  of  collisions  and  either  the 
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Figure  4.  Overhead  view  of  the  Turns  task. 


niomber  of  laps  completed  within  a  time  limit  or  the  time  to 
complete  an  arbitrary  number  of  laps . 

The  intersection  where  the  Figure- 8  course  crosses  over  itself 
presents  a  complex  visual  scene.  Figure  2  is  a  photograph  of  a 
monitor  showing  the  intersection  from  the  participant's  perspec¬ 
tive.  Three  paths  are  visible:  a  sharp  turn  to  the  left  (incor¬ 
rect)  ,  a  gradual  turn  to  the  left  (correct) ,  and  a  gradual  turn 
to  the  right  (incorrect) . 


Doorways  (Figure  5! 


The  Doorways  task  represents  a  VE  "road  test"  of  the  kind 
and  difficulty  of  walking  performance  that  might  be  required  in  a 
VE  training  application.  The  course  is  formed  by  a  series  of 
rooms  connected  by  7  by  3  ft  doorways.  The  positions  of  the  doors 
in  the  walls  vary  so  that  a  series  of  non- 90  degree  turns  must  be 
made  to  navigate  the  course  efficiently. 
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Two  flying  tasks  were  developed  for  a  different  purpose  than 
that  of  the  walking  tasks.  Flying  could  be  used  during  mission 
planning  to  move  rapidly  from  one  place  to  another  or  to  gain  a 
top-down  view  of  a  physical  space. 

Windows  (Figure  6) 


This  task  requires  lateral,  horizontal,  and  vertical 
("flying")  movement  through  a  series  of  ten  rooms.  On  the  far 
wall  of  each  room  is  a  7-ft  x  3-ft  window  that  provides  the  only 
access  into  the  next  room.  Windows  vary  in  terms  of  both  their 
vertical  (top,  center,  or  bottom)  and  horizontal  (left,  middle, 
or  right)  positions. 

Elevator  (Figure  7) 

This  task  involves  "flying"  over  and  under  a  series  of  nine 
vertical  partitions  while  moving  forward.  Efficient  performance 
requires  the  participant  to  look  up  and  down  while  moving.  There 
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Figure  6.  A  perspective  view  of  the  Windows  task  (walls,  ceiling, 

and  floor  have  been  removed). 


are  nine  vertical  turns  in  this  task. 

Tracking  Tasks 

The  tracking  tasks  measure  skill  in  controlling  the  position 
of  a  cursor  relative  to  stationary  and  moving  targets.  Two  modes 
of  pursuit  tracking  are  employed.  In  the  HMD  control  mode  the 
aiming  cursor,  a  white  cross  hair,  appears  at  the  center  of  the 
field  of  view.  The  cursor  is  slaved  to  the  center  of  the  field 
of  view,  so  that  participants  are  required  to  use  head  movement 
to  track  the  target.  The  HMD  mode  was  designed  to  measure  the 
ability  to  control  the  direction  and  steadiness  of  head  movement. 
In  the  control  device  mode  the  cursor  is  aimed  by  a  hand- 
controlled  input  device.  The  two  modes  of  cursor  control  and  two 
modes  of  target  movement  were  combined  to  form  four  tasks:  HMD 
mode,  stationary  target:  HMD  mode .  moving  target :  Controller 
mode,  stationary  target;  and  Controller  mode,  moving  target.  The 
target,  a  ball  .7  ft  (8.4  in.)  in  diameter,  appears  on  a  wall  10 
ft  away  (Figure  8) .  In  the  moving- target  mode,  the  ball  moves 
across  the  wall's  surface  at  about  1.4  ft  per  second.  The  target 
changes  color  when  the  cursor  is  within  the  target  radius.  The 
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Figure  8.  A  participant’s  view  of  the  Tracking  tasks. 
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target  disappears,  ending  the  trial,  upon  reaching  another  wall 
or,  for  the  stationary- target  mode,  when  20  seconds  has  elapsed. 
Random  numbers  determine  the  placement  of  stationary  targets  and 
the  direction  of  moving  targets.  The  performance  measure  is  the 
percentage  of  the  total  trial  time  during  which  the  cursor  is  on 
the  target,  i.e.,  time  on  target. 

Object  Manipulation  Tasks 

The  participant  uses  a  control  device  to  move  a  cursor  which 
interacts  with  objects  in  the  VE.  The  position  of  the  cursor  in 
the  VE  is  shown  as  a  3-D  cross.  When  the  cursor  is  in  contact 
with  a  manipulable  object  the  color  of  that  object  changes. 
Pushing  a  button  on  the  control  device  'grasps'  the  object;  a 
successful  grasp  is  indicated  by  an  additional  change  in  the 
object's  color.  A  grasped  object  will  move  with  the  cursor. 
Performance  measures  are  whether  the  task  is  successfully  com¬ 
pleted  within  a  designated  time  limit  and  the  time  required  to 
complete  the  task. 

Slide  (Figure  9) 

Grasp  an  object,  similar  to  a  slider  bar  on  a  control  panel, 
and  move  it  horizontally  to  a  target  location.  A  30- second  time 
limit  is  enforced  for  each  trial. 

Dial  (Figure  10) 

Grasp  a  dial,  similar  to  a  volume  control  on  a  radio,  and 
rotate  it  to  a  target  orientation.  A  30 -second  time  limit  is 
enforced  for  each  trial . 
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Figure  10.  A  participant’s  view  of  the  Dial  task. 


18 


Bins  (Figure  11) 


The  participant  faces  a  3  by  3  stack  of  open-ended,  box- like 
compartments  (bins) .  At  the  beginning  of  a  trial  a  ball  appears 
in  one  of  the  bins,  an  X  appears  in  another.  The  cursor  is  used 
to  grasp  and  drag  the  ball  out  of  the  bin  and  into  the  bin  marked 
with  an  X.  A  45 -second  time  limit  is  enforced  for  each  trial. 
Figure  11  depicts  the  beginning  of  a  trial .  The  ball  is  in  the 
left  column  of  the  middle  row  of  bins,  the  cursor  is  slightly  in 
front  of  the  center  bin,  and  an  X  marks  the  target  bin;  the 
middle  bin  in  the  top  row. 


Figure  11 .  A  participant’s  view  of  the  Bins  task. 


Reaction  Time  Tasks 


Simple  and  choice  reaction  time  tasks.  Simple  RT  and  Choice 
RT  (Figure  12) ,  were  developed  to  measure  response  time  to  VE 
events  and  to  provide  an  indication  of  the  time  lag  of  the  VE 
system  in  presenting  and  recording  events .  For  Simple  RT  the 
participant  pushes  the  control  device  as  soon  as  a  black  X 
appears  against  a  white  background.  For  Choice  RT  the 
participant  pushes  the  control  device  in  the  appropriate 
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direction  to  indicate  in  which  of  four  windows  an  X  appears. 
Performance  measures  are  the  percentage  of  correct  responses  and 
time  to  respond. 


Figure  1 2.  A  participant’s  view  of  the  Choice  Reaction  Time  task. 

Evaluation  of  VEPAB 


Our  initial  evaluation  of  the  VEPAB  focused  on  the 
sensitivity  of  the  tasks  to  differences  between  two  input  control 
devices  and  differences  among  individual  participants.  If 
changing  the  input  interface  does  not  result  in  a  change  in  task 
perfoinnance,  then  that  task  would  be  of  no  value  in  comparing 
system  components.  This  might  happen  if  the  task  was  too 
difficult  or  too  easy,  if  the  performance  measures  were 
insufficiently  sensitive  to  detect  such  differences,  if 
performance  was  largely  determined  by  factors  unrelated  to  the 
interface,  or  if  performance  was  artificially  limited.  We  were 
also  interested  in  how  task  performance  varied  as  a  function  of 
practice.  Lack  of  a  practice  effect  could  result  from  any  of  the 
reasons  listed  above  or  if  the  task  fails  to  provide  appropriate 
feedback. 
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A  second  experiment  is  planned  in  which  we  will  compare 
VEPAB  performance  across  different  visual  display  devices.  That 
experiment  will  provide  an  indication  of  the  sensitivity  of  the 
VEPAB  vision  tasks.  However,  in  this  experiment  we  did  collect 
real-world  vision  performance  data  to  provide  a  comparison  with 
performance  on  the  VEPAB  vision  tasks.  In  addition  to  providing 
comparison  data  Snellen  acuity  and  color  vision  were  tested 
before  and  after  VE  immersion  to  check  for  aftereffects.  These 
tests  also  served  to  screen  out  participants  with  abnormal 
vision.  We  collected  real-world  data  for  the  distance  estimation 
task  because  of  our  concern  that  the  procedure  itself,  not  the 
visual  properties  of  VE,  might  lead  to  systematic  error.  These 
real-world  distance  estimation  task  data  were  inexpensive  and 
easy  to  collect.  In  contrast,  the  expense  of  collecting  real- 
world  data  for  the  search,  locomotion,  and  manipulation  tasks  did 
not  seem  warranted.  Some  tasks  do  not  have  real-world 
counterparts  (search  or  the  flying  tasks,  for  example) .  Others 
would  have  required  special  fabrication  of  equipment  (bins  and 
head  tracking,  for  example) . 


Method 


Participants 

Twenty- four  research  participants  completed  each  of  the 
tasks  shown  in  Table  1.  The  participants,  twenty- one  students 
and  three  government  employees,  had  normal  or  corrected- to-normal 
vision.  The  age  of  the  participants  varied  from  17  to  37  years 
with  a  mean  of  24.  Sixteen  were  male,  and  8  were  female. 
Estimated  average  weekly  computer  use  varied  from  0  to  20  hours 
with  a  mean  of  3.7.  Average  weekly  video  game  use  varied  from  0 
to  26  hours  with  a  mean  of  6.4.  The  students  were  paid  for  their 
participation. 

Apparatus 

A  Virtual  Research  Flight  Helmet  with  83  degree  horizontal 
and  41  degree  vertical  field  of  view  (50  horizontal  by  41 
vertical  for  each  eye)  provided  the  visual  display.  The  Flight 
Helmet  displays  234  lines  of  238  pixels  to  each  eye  and  uses  LCD 
technology  with  LEEP  optics.  Stereoscopic  images  were  generated 
by  two  linked  IBM  Compatible  486/DX50  mhz  PCs  equipped  with  Intel 
ActionMedia  graphics  boards.  Each  computer  provided  input  to  one 
visual  channel .  A  Polhemus  Isotrack  provided  head  tracking  of 
yaw,  pitch,  and  roll.  Participants  controlled  their  movement  in 
and  interaction  with  the  VEs  with  either  a  Silicon  Graphics 
Spaceball  or  a  Gravis  Joystick.  The  VEs  were  developed  with 
Sense8  WorldToolKit  software.  Two  standard  monitors  provided  the 
experimenter  with  a  representation  of  the  participant's  view  of 
the  visual  channels  of  the  HMD.  A  menu  system  allowed  the 
experimenter  to  select  tasks  in  any  order,  start  and  terminate 
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trials,  turn  head  tracking  on  or  off,  and  reset  the  participant's 
view. 

Procedure 


Overview.  Concern  for  potential  side  effects  and 
aftereffects  strongly  influenced  the  experimental  procedure. 
Participants  were  warned  that  they  might  experience  nausea, 
eyestrain,  or  dizziness  and  were  encouraged  to  call  a  "timeout" 
at  the  onset  of  any  of  those  symptoms.  After  each  twenty  minutes 
of  VE  activities  participants  removed  the  HMD  for  a  short  break. 
Participants  remained  seated  in  a  swivel  chair  while  wearing  the 
HMD  to  reduce  the  danger  of  falling.  The  HMD  occluded  much  of  the 
participant ' s  face  preventing  the  experimenter  from  detecting 
facial  indications  of  discomfort.  Therefore,  we  required  that 
participants  speak  frequently  throughout  the  experiment  to 
indicate  that  they  were  ready  to  continue  the  tasks.  Because  we 
had  concerns  about  aftereffects  altering  participants'  ability  to 
drive  an  automobile  safely,  we  required  them  to  remain  at  the 
experiment  site  for  at  least  one  hour  after  the  VE  tasks  had  been 
completed.  During  this  time  they  completed  questionnaires. 

One-half  of  the  participants  performed  the  motor  tasks  using 
a  spaceball  as  the  control  device,  and  the  other  half  used  a 
joystick.  They  were  familiarized  with  the  appropriate  control 
device,  the  HMD,  and  how  to  recognize  collisions  but  did  not 
practice  any  specific  task  prior  to  data  collection.  A  wooden 
pedestal  supported  the  control  device  at  hand  level.  The  room 
temperature  was  70  degrees.  The  tasks  were  performed  in  two 
sessions  on  separate  days. 

Day  1.  The  participant  read  an  overview  of  the  purpose  and 
procedure  of  the  experiment  then  signed  the  consent  form  and 
completed  a  background  information  questionnaire  concerning  age 
and  previous  experience  with  computers  and  video  games.  The 
participant  completed  all  of  the  vision  tasks  and  the  locomotion 
tasks  in  the  order  shown  in  Table  1  up  to  and  including  the 
"Doorways"  task.  The  e3q)erimenter  explained  and  demonstrated  the 
operation  of  the  control  device  before  the  participant  began  the 
locomotion  tasks.  Real-world  tests  of  Snellen  acuity,  color 
perception,  contrast  sensitivity,  and  stereopsis  were 
administered  both  before  and  after  the  participant  performed  the 
VE  tasks.  These  tests  were  administered  to  test  for  aftereffects 
resulting  from  immersion  in  VEs.  In  addition,  questionnaires 
addressing  simulator  sickness  (Appendix  A)  and  presence  (Appendix 
B)  were  administered  after  the  VE  immersion  ended. 

Day  2.  Beginning  with  the  "Windows"  task,  the  participants 
completed  the  remaining  tasks  in  the  order  listed  in  Table  1.  To 
test  for  aftereffects  a  mirror  tracing  task  was  administered  both 
before  and  after  the  participant  performed  the  VE  tasks.  The 
task  involved  tracing  within  double  lines  which  formed  a  star 
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pattern.  The  participant  did  not  see  the  pattern  directly  but 
rather  saw  the  mirror  reflection  of  the  pattern.  In  addition,  the 
questionnaires  addressing  simulator  sickness  and  presence  were 
administered  after  the  VE  immersion  ended. 

Results  and  Discussion 


The  results  section  begins  with  the  presentation  of 
Cronbach's  Alpha  measure  of  reliability  (Cronbach,  1951)  for  each 
of  the  VEPAB  tasks  in  which  there  were  repeated  trials.  The 
results  of  the  administration  of  the  vision  tasks  are  described. 
The  results  of  the  analyses  to  address  the  primary  focus  of  the 
experiment,  whether  the  tasks  were  sensitive  to  differences 
between  control  devices  and  to  practice  effects,  are  presented. 
The  relationships  among  performance  on  the  various  tasks,  and 
among  task  performance,  background  characteristics,  and  side 
effects  are  presented.  Finally,  the  aftereffects  resulting  from 
immersion  in  VEs  are  described. 

Although  all  participants  who  had  participated  in  the  first 
session  agreed  to  return  for  the  second,  four  did  not.  For  one 
participant  the  data  file  of  second  session  performance  was 
unusable.  In  addition,  the  system  occasionally  failed  to  capture 
data  for  some  trials.  Therefore,  not  all  of  the  analysis  are 
based  upon  complete  data  sets  for  all  participants. 

Reliability 

Table  2  lists  the  Cronbach's  Alpha  measure  of  reliability 
for  each  of  the  tasks  in  which  there  were  repeated  trials.  For 
several  of  the  tasks,  reliability  coefficients  could  not  be 
computed  for  accuracy  scores.  The  Search  and  Simple  RT  tasks 
were  always  performed  correctly,  and  consequently,  there  was  no 
variation  in  accuracy  scores.  Accuracy  data  for  the  Turns,  Door¬ 
ways,  Windows,  and  Elevator  tasks  were  not  captured  at  the 
individual  trial  level . 
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Table  2 


VEPAB  Task  Reliability  (Cronbach's  Alpha) 


Task 

N 

Reliability  (Alpha) 

Time 

Accuracy 

Search 

23 

.19 

N.A. 

Straightaway 

20 

.85 

.48 

Backup 

22 

.96 

.06 

Turns 

23 

.97 

N.A. 

Doorways 

23 

.75 

N.A. 

Windows 

20 

.68 

N.A. 

Elevator 

19 

.89 

N.A. 

Bins 

19 

.85 

.82 

Dial 

19 

.88 

.87 

Slider 

19 

.83 

.67 

Simple  RT 

19 

.24 

N.A. 

Choice  RT 

19 

.75 

.42 

Tracking  -  Head  Control, 
Stationary  Target 

19 

.43 

.47 

Tracking  -  Head  Control, 
Moving  Target 

19 

.88 

.92 

Tracking  -  Device  Control, 
Stationary  Target 

18 

.69 

.66 

Tracking  -  Device  Control, 
Moving  Target 

18 

.69 

.65 

Note.  For  Tracking  tasks,  the  time  measure  was  percent  time  on 
target,  and  the  accuracy  measure  was  the  average  distance  of  the 
cursor  from  the  target. 


Task  reliability  was  generally  high,  but  there  were  some 
exceptions.  Very  low  reliability  was  found  for  the  accuracy 
measure  of  the  Backup  task,  and  low  reliability  for  the  accuracy 
measure  of  the  Straightaway  task.  Both  may  reflect  a  problem 
with  the  accuracy  measures  for  the  locomotion  tasks  related  to 
the  way  in  which  the  data  capture  system  counted  collisions.  A 
collision  was  counted  for  each  frame  in  which  the  simulated  body 
was  in  contact  with  a  wall  or  door  frame.  This  sometimes 
resulted  in  dozens  of  collisions  being  counted  for  what  seemed  to 
the  participant  and  the  experimenter  to  be  a  single  collision. 

The  Search  task  also  had  low  reliability.  We  believe  that  the 
task  difficulty  varied  from  trial  to  trial  as  a  function  of  the 
starting  position  of  the  target. 

The  other  tasks  with  low  reliability  scores  were  Simple  RT 
(time)  and  Choice  RT  (accuracy) .  For  Simple  RT  the  slow  system 
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update  rate  (approximately  90  ms  for  this  task)  probably 
overshadowed  any  individual  differences  in  reaction  time.  For 
some  participants  the  accuracy  scores  for  Choice  RT  probably 
reflected  a  problem  with  the  stimulus/response  compatibility  in 
that  the  up  and  down  responses  were  occasionally  confused. 

Vision 


Acuity.  From  a  simulated  distance  of  20  feet  the 
participants  reported  a  uniform  white  rectangle.  At  ten  feet 
participants  reported  a  picture  with  geometric  shapes.  The  mean 
distance  at  which  participants  could  first  recognize  the  eye 
chart  and  read  the  top  line  (the  big  "E")  of  the  chart  was  4.65 
ft,  for  about  20/860  acuity.  The  distance  from  which  the  "E"  was 
first  recognized  ranged  from  6  ft  (20/666)  to  2  ft  (20/2000) . 

In  contrast,  visual  acuity  of  approximately  20/250  would  be 
predicted  based  solely  upon  the  pixel  density  of  the  HMD.  The 
discrepancy  between  the  theoretical  resolution  of  the  HMD  and  the 
acuity  values  we  observed  may  have  resulted  from  several  factors. 
Imperfections  in  the  modelling  of  the  individual  letters  and  in 
the  image - rendering  software  probably  interact  so  that  some  image 
degradation  occurs  before  the  image  is  displayed  in  the  HMD.  In 
addition,  physical  variation  across  participants,  such  as 
interpupillary  distance,  may  lead  to  variation  in  acuity. 

Robinett  and  Rolland  (1992)  described  the  complex  challenge  of 
computing  correct  stereoscopic  images  for  HMDs. 

Acuity  was  initially  measured  with  head  tracking  off,  then 
checked  with  head  tracking  on.  In  the  head  tracking  off 
condition  the  field  of  view  of  the  VE  is  not  coupled  to  the 
orientation  of  the  HMD.  Half  of  the  participants  stated  that 
their  acuity  was  better  with  head  tracking  off,  the  other  half 
preferred  head  tracking  on.  However,  no  difference  in  acuity  was 
detected  as  a  function  of  head  tracking  being  on  or  off. 

Color.  All  participants  had  normal  color  vision  as 
indicated  by  maximum  scores  on  the  real-world  color  vision  tests. 
The  first  sixteen  participants  also  had  maximum  scores  on  the  VE 
color  vision  tests.  However,  for  the  last  eight  participants, 
not  all  of  the  numerals  were  recognized.  We  interpreted  this  as 
an  indication  that  the  HMD  color  display  had  weakened  over  time. 

Object  recognition,  size  estimation,  range  estimation.  All 
participants  recognized  the  object  at  the  end  of  the  corridor  as 
a  hximan  figure.  The  mean  height  estimate  was  62.83  in.  (S.D.= 
8.27),  approximately  9  in.  shorter  than  the  actual  height  of  the 
figure.  This  may  be  a  function  of  the  VE  ceiling  height  of  12 
ft.  Participants  may  have  assxamed  that  the  ceiling  was  about  10 
ft  high.  The  six  ft  human  figure  was  exactly  one -half  of  the 
height  of  the  corridor.  One-half  of  a  ten  ft  corridor  would  be 
60  in.,  fairly  close  to  the  mean  estimate. 
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At  an  actual  range  of  40  ft,  the  average  range  estimate  was 
38.22  ft,  S.D.  =  21.70,  with  a  minimum  of  9.00  and  a  maximum  of 
100.00.  Thus  at  this  distance,  the  average  distance  estimate  in 
VE  was  accurate.  However,  note  the  range  listed  above. 
Paradoxically,  as  the  figure  approached  the  participant's  eye 
point,  and  presumably  stereoscopic  depth  cues  should  be  more 
effective,  the  VE  distance  estimates  became  less  accurate.  This 
may  be  function  of  the  difference  between  each  participant's  real 
world  standing  eye  height  and  their  VE  eye  point.  VE  eye  point 
was  set  at  a  height  of  64  in.  for  all  participants,  regardless  of 
real-world  eye  height.  As  the  figure  approached  the  participant, 
any  cues  to  distance  provided  by  relative  height  of  the  figure 
would  become  increasingly  inaccurate. 

The  VEPAB  acuity  task  was  modeled  after  a  standard  vision 
task  for  which  real-world  baseline  data  are  readily  available. 

For  the  VE  distance  estimation  task,  unlike  the  acuity  task,  we 
did  not  have  real-world  data.  In  addition,  we  wanted  to  confirm 
that  the  procedure  itself  did  not  lead  to  systematic  error  in  the 
estimation  of  distance.  Therefore,  we  recreated  the  VE  distance 
estimation  task  in  the  real-world.  That  is,  a  hviman  moved  down  a 
corridor  similar  to  that  portrayed  in  the  VE.  A  different  set  of 
participants  estimated  distances  following  the  same  method  used 
in  the  VE  task. 

Figure  13  depicts  the  means  for  the  VE  and  real-world 
estimates  of  distance  plotted  against  the  actual  distance.  In 
this  figure  the  X  axis  value  labeled  "40"  indicates  that  the 
figure  was  forty  ft  away  when  the  participant  was  first  asked  to 
estimate  the  distance  to  the  figure.  The  Y  axis  value  presents 
the  participant's  estimate  of  the  distance  when  the  figure  was 
actually  40  ft  away.  In  contrast,  the  other  X  axis  values  (30, 
20,  etc.)  correspond  to  the  distances  at  which  the  participant 
was  supposed  to  call  out  as  the  moving  figure  reached  those 
distances  in  approaching  the  participant.  For  example,  the  Y 
axis  represents  the  actual  distance  to  the  figure  when  the 
participant  estimated  the  distance  to  be  30  ft.  In  other  words, 
except  for  the  value  at  40  ft  the  Y  axis  values  represent  the 
actual  distance  of  the  figure. 

In  the  VE,  the  average  distance  estimate  at  40  ft  was  very 
accurate;  however,  variance  was  high.  VE  distance  estimation  was 
less  accurate  at  shorter  distances,  contrary  to  our  expectations 
that  stereoscopic  depth  cues  should  assist  range  estimation  at 
shorter  distances.  Although  real-world  distance  estimation  was 
poor  at  40  ft,  estimates  at  the  other  distances  were  accurate. 
Therefore,  we  do  not  believe  that  inaccurate  estimation  of 
distance  in  the  VE  is  merely  an  artifact  of  our  distance 
estimation  procedure. 
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Figure  13.  Virtual  and  real-world  distance  estimates  as  a  function 

of  actual  distance. 

Search.  All  participants  were  able  to  perform  this  task, 
which  required  extensive  head  movement.  However,  performance 
varied  greatly  across  participants.  The  mean  search  time  per 
participant,  averaged  across  10  trials,  was  10.4  seconds  with  a 
minimum  of  5.5  and  a  maximum  of  24 .  We  observed  that  some 
participants  adopted  a  systematic  search  strategy,  slowly  panning 
up  and  down  the  walls  while  turning  in  the  swivel  chair,  but 
others  seemingly  unsystematically  slewed  their  point  of  view 
around  the  room.  We  did  not  have  a  procedure  to  categorize  the 
search  strategy  of  each  participant  and  therefore  we  can  not 
compare  performance  differences  across  strategies. 

Control  Device  and  Practice  Effects 

For  each  task,  a  separate  repeated  measures  analysis  of 
variance  was  conducted  on  the  time  scores.  For  these  analyses, 
control  device  (Device)  was  the  between-participants  factor,  and 
practice  was  the  within-participants  factor.  The  practice 
effects  for  each  of  the  locomotion  tasks  are  based  on  the  mean 
completion  time  of  the  first  five  segments  versus  the  mean  of  the 
last  five  segments.  For  the  other  tasks  the  mean  completion  time 
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of  the  first  five  trials  was  compared  with  the  mean  of  the  last 
five  trials.  For  several  of  the  tasks  similar  analyses  were  also 
performed  on  the  accuracy  scores. 

Locomotion  tasks.  Table  3  summarizes  the  ANOVAs  for  the 
locomotion  tasks.  For  each  ANOVA  the  table  lists  the  F  value 
(indicating  the  significance  level)  and  the  partial  ETA  squared 
index  of  effect  size.  For  each  of  the  locomotion  tasks  the  group 
effect  (joystick  vs  spaceball)  of  an  ANOVA  on  the  time  measure 
was  significant.  Figure  14  shows  the  mean  completion  time  for 
each  segment  of  the  locomotion  tasks  as  a  function  of  Control 
Device.  For  all  of  these  tasks  performance  with  the  joystick  was 
faster  (better)  than  the  spaceball. 

We  believe  that  there  were  several  reasons  why  joystick 
performance  was  superior  to  that  with  the  spaceball.  We  suspect 
that  our  participants  were  more  familiar  with  the  joystick  than 
the  spaceball  prior  to  the  experiment.  Joysticks  are  a  common 
component  of  video  games;  spaceballs  are  not.  Also,  the  display 
was  updated  relatively  slowly,  approximately  2  to  9  times  per 
second,  depending  on  the  task.  The  spaceball  did  not  move 
perceptibly  when  force  was  applied,  and  this  may  have  interacted 
with  the  slow  update  to  make  it  particularly  difficult  to  learn 
to  use.  The  same  results  might  not  be  obtained  if  the  effects  of 
applying  force  to  the  spaceball  were  immediately  apparent.  In 
particular,  the  lag  between  when  a  participant  applied  force  to 
the  spaceball  and  the  time  when  this  action  produced  a  noticeable 
effect  led  participants  to  the  erroneous  and  counterproductive 
belief  that  they  needed  to  apply  additional  force  to  the 
spaceball.  We  noticed  that  several  participants  exhibited  "white 
knuckles"  as  they  struggled  to  apply  force  to  the  spaceball. 

Only  two  of  the  locomotion  tasks  showed  significant  practice 
effects:  Straightaway,  the  first  locomotion  task  performed,  and 

Elevator,  the  second  of  the  two  "flying"  tasks.  Performance 
improved  with  practice  for  both  of  these  tasks.  The  Straightaway 
task  had  a  significant  interaction  of  group  and  practice.  Figure 
15  presents  the  group  mean  completion  times  for  the  first  five 
segments  and  second  five  segments  of  the  Straightaway  task.  This 
figure  indicates  that  spaceball  performance  was  worse  than 
joystick  performance  on  the  first  five  trials  and  that  although 
both  groups  improved  with  practice,  the  improvement  was  greater 
with  the  spaceball  group.  The  Elevator  task  has  only  9  segments, 
therefore  to  examine  the  practice  effect  we  compared  the  mean 
completion  time  for  the  first  4  segments,  which  was  41  seconds, 
with  the  mean  of  last  4  segments,  18  seconds. 
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Table  3 


F  Values  From  the  ANOVAS  of  Time  and  Accuracy  Scores  for  Each 

Locomotion  Task 
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Straightaway 

Back-up 

Turns 

Doorways 

Figure  8 

Windows 

Elevator 

Joystick 

14.53  8.11 

13.01 

13.89 

11.58 

41.7 

11.98 

Spaceball 

23.54  16.11 

24.87 

21.53 

20.5 

85.09 

32.27 

Figure  14.  Locomotion  task  segment  completion  time 
as  a  function  of  control  device. 


1st  half  2nd  half 


Joystick  78.6  66.5 

Spaceball  150  85.3 

Figure  15.  Straightaway  task  completion  time  as  a  function  of 
control  device  and  practice. 
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The  lack  of  a  significant  practice  effect  for  some  of  the 
locomotion  tasks  may  have  resulted  from  a  problem  with  the  tenth 
segment  in  those  tasks.  The  Turns,  Doorways,  and  Windows  tasks 
had  a  "finish  line",  consisting  of  multi-colored  blocks,  that 
defined  the  end  of  the  course.  The  participants  were  told  what 
the  blocks  represented  and  were  instructed  to  move  through  the 
blocks  without  hesitation.  Nevertheless,  some  participants 
seemed  to  pause  when  approaching  the  blocks.  In  addition,  the 
blocks  presented  a  confusing  visual  pattern  such  that  the  final 
course  segment  lacked  some  of  the  depth  cues  that  had  been 
visible  in  the  preceding  segments  of  the  course. 

Another  factor  that  may  have  obscured  practice  effects  is 
that  some  of  the  participants,  gaining  confidence  during  the 
later  trials  of  a  task,  would  experiment  with  moving  at  higher 
speeds  which  led  to  more  collisions  and  therefore  slower 
completion  times  for  the  later  segments.  Also,  the  ten  segments 
or  trials  of  each  task  provide  only  limited  practice. 

Table  4 


F  Values  From  ANOVAs  of  Time  and  Accuracy  Scores  for  Each 
_ Manipulation  Task _ 


Task 

Factor 

F 

Time 

Accuracy 

Bins 

Device 

17.77** 

7.27* 

Practice 

36.98** 

26.46** 

DxP 

.76 

.45 

Dial 

Device 

8.85** 

12.06** 

Practice 

7.66 

4.17 

DxP 

.01 

.00 

Slider 

Device 

49.06** 

21.98** 

Practice 

30.15** 

6.94* 

DxP 

.58 

1.03 

*P<.05  **p<.01 


Manipulation  tasks.  Table  4  summarizes  the  ANOVAs  for  the 
manipulation  tasks.  For  each  task  the  group  effect  was  signifi¬ 
cant  for  both  the  time  and  accuracy  scores.  For  each  of  the 
three  tasks,  significant  practice  effects  were  found  for  comple- 
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Practice 

□  1st  half  ■2nd  half 


Slider 

Dial 

Bins 

1  St  half 

74 

64 

59 

2nd  half 

88 

78 

82 

Figure  16.  Manipulation  task  accuracy  (percent  successful 
trials)  as  a  function  of  practice. 

tion  time  measures.  The  practice  effects  for  accuracy  measures 
were  significant  for  the  Bins  and  Slider  tasks,  and  approached 
significance  (p=.057)  for  the  Dial  task.  See  Figures  16  and  17 
for  the  accuracy  and  completion  time  means,  respectively,  for  the 
practice  effect. 

Figure  18  presents  the  mean  trial  completion  time  as  a 
function  of  control  device  for  each  of  the  manipulation  tasks. 
Figure  19  shows  performance  accuracy,  the  percentage  of 
successful  trials,  for  each  task  as  a  function  of  control  device. 
Joystick  performance  was  superior  for  both  speed  and  accuracy 
measures . 

Tracking .  Separate  analyses  of  variance  were  performed  for 
the  two  different  tracking  modes:  cursor  aimed  by  the  head 
position  tracker  on  the  HMD  and  cursor  aimed  by  the  manual 
control  device  (Spaceball  or  Joystick) .  The  analyses  are 
summarized  in  Table  5.  The  duration  of  the  tracking  trials 
varied,  therefore  analyses  are  based  on  the  percent  of  time  on 
target,  that  is,  the  ratio  of  the  time  that  the  cursor  was  on  the 
target  to  the  total  time  that  the  target  was  visible.  The 
Locomotion  and  Manipulation  tasks  showed  sensitivity  to  control 
devices ,  but  the  Tracking  tasks  did  not .  We  suspect  that  the 
slow  system  update  rate,  approximately  200  ms  for  the  tracking 
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Slider 

Dial 

Bins 

2nd  half 

12.19 

13.58 

25.37 

1st  half 

17.3 

17.32 

33.52 

Figure  1 7.  Manipulation  task  completion  time  as  a  function  of 

practice. 


^  Slider  Dial  Bins 

Joystick  10.34  11.25  23.88 

Spaceball  18.7  19.24  34.45 


Figure  1 8.  Manipulation  task  completion  time  as  a  function  of 

control  device. 
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Control  Device 
Spaceball  □Joystick 


Slider  Dial  Bins 


Spaceball  68  53  51 

Joystick  95.56  91.11  88.56 

Figure  19.  Manipulation  task  accuracy  (percent  successful  trials) 

as  a  function  of  control  device 


tasks,  made  the  tasks  so  difficult  that  they  could  not  be 
performed  well  with  any  control  device.  Tracking  attempts 
seesawed  between  lagging  and  overshooting  the  target  so  that 
time -on- target  scores  were  very  poor;  participants  were  able  to 
keep  the  cursor  on  the  moving  target  less  than  9%  of  the  time. 

As  expected,  stationary  targets  were  significantly  easier  to 
track  than  moving  targets  in  both  HMD  and  manual  tracking  (Figure 
20) .  Manual  tracking  improved  with  practice  (Figure  21) ,  HMD 
tracking  did  not.  We  believe  that  the  lack  of  a  significant 
practice  effect  in  HMD  tracking  was  in  part  the  result  of  a 
fatigue  effect;  during  the  later  trials  in  the  head  tracking  mode 
some  participants  grasped  and  guided  the  HMD  with  their  hands 
apparently  because  their  neck  muscles  were  fatigued. 
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Table  5 


F  Values  From  ANOVAs  of  Time  on  Target  Scores  for  Each  Tracking 

Task 


Task 

Factor 

£ 

HMD  control 

Target 

565.90“ 

Practice 

.13 

TxP 

1.33 

Manual  control 

Device 

.  16 

Target 

308.31“ 

Practice 

16.77“ 

DxT 

.15 

DxP 

.22 

TxP 

16.32“ 

DxTxP 

3.11 

*E<.05  **e<.01 


□  HMD  Tracking  ■  Device  Tracking 


Moving  Stationary 


HMD  Tracking  8  58 

Device  Tracking  9.72  58.49 

Target  Movement 

Figure  20.  Tracking:  Percent  time  on  target  as  a  function 
of  target  movement  and  type  of  control. 


1st  5  trials  2nd  5  trials 

30.98  37.24 

Practice 


Figure  21 .  Manual  tracking:  Time  on  target  as  a  function 
of  practice. 
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For  manual  tracking  there  was  a  significant  interaction  of 
target  movement  and  practice.  Figure  22  shows  that  the  percent¬ 
age  of  time  on  target  improved  with  practice  for  stationary 
targets  but  not  for  moving  targets.  We  suspect  that  the  slow 
system  update  rate  made  tracking  moving  targets  so  difficult  that 
the  effects  of  limited  practice  were  overshadowed. 


Figure  22.  Manual  tracking:  Interaction  of  target  movement  and 

practice 


Reaction  Time.  The  results  of  the  analysis  of  variance  of 
the  reaction  time  data  are  shown  in  Table  6.  There  were  no 
significant  Device  effects  for  the  reaction  time  tasks.  An 
explanation  is  that,  unlike  the  other  motor  tasks,  success  of  the 
reaction  time  tasks  did  not  require  the  participants  to  vary  the 
range  of  movement  of  the  control  device.  Therefore,  the  factors 
that  we  believe  interacted  to  degrade  performance  for  the 
spaceball  in  other  tasks,  lack  of  perceptible  movement  of  the 
controller  and  slow  system  update  rate,  did  not  affect 
performance  on  the  reaction  time  tasks. 
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Table  6 


F  Values  From  ANOVAs  of  Time  and  Accuracy  Scores  for  Each 

Reaction  Time  Task 


Task 

Factor 

F 

Time 

Accuracy 

Simple 

Device 

.01 

N.A. 

Practice 

5.62* 

N.A. 

DxP 

.02 

N.A. 

Choice 

Device 

2.46 

1.58 

Practice 

7.96* 

2.87 

DxP 

.21 

.06 

*E<.05  **e<.01 


There  were  significant  practice  effects.  Figure  23  presents 
the  means  for  the  Simple  and  Choice  reaction  time  tasks  as  a 
function  of  practice.  The  reaction  times  tasks  were  the  only 
motor  tasks  for  which  we  did  not  expect  significant  practice 
effects.  The  experimenter  may  have  inadvertently  established  a 
temporal  pattern  such  that  the  participants  could  anticipate  the 
presentation  of  the  reaction  time  stimuli. 

The  mean  for  simple  reaction  time  (collapsed  across  Device) 
was  .36  seconds  and  the  mean  for  choice  reaction  time  was  .50 
seconds.  McCormick  &  Sanders  (1976)  list  .2  seconds  as  a  "fairly 
representative"  value  of  simple  reaction  time.  Relative  to  what 
would  be  expected  in  similar  real-world  tasks,  the  slowness  of  VE 
reaction  time  probably  reflects  lags  in  the  VE  system  in 
presenting  the  stimulus  and  responding  to  activation  of  the 
control  device. 

Correlations  among  Variables 

Because  ANOVAs  had  indicated  significant  differences  between 
the  joystick  and  spaceball  control  devices,  we  derived  partial 
correlation  coefficients,  with  Device  as  the  control  variable,  to 
examine  the  relationships  among  performance  on  the  various  tasks, 
and  between  task  performance  and  other  variables.  Because  the 
time  scores  were  the  more  reliable  measure,  we  used  them  rather 
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than  the  accuracy  scores.  We  examined  the  correlations  among  all 
of  the  locomotion,  manipulation,  tracking,  and  reaction  time 
tasks,  and  between  each  of  those  tasks  and  other  information 
(age,  mirror  tracing  performance,  virtual  visual  acuity,  number 
of  hours  of  weekly  computer  use,  and  number  of  hours  of  weekly 
video  game  playing) .  The  purpose  of  these  analyses  was  to 
identify  possible  patterns  of  correlations  for  confirmation  in 
subsequent  research.  We  did  not  adjust  significance  levels  to 
take  into  account  the  total  number  of  correlations  computed;  to 
do  so  would  result  in  prohibitively  conservative  values  for 
explanatory  purposes. 
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Figure  23.  Simple  and  choice  reaction  time  as  a  function  of 

control  device 


There  were  no  significant  correlations  of  computer  use, 
video  game  use,  or  virtual  visual  acuity  with  the  score  on  the 
VEPAB  tasks.  The  correlations  with  age,  mirror  tracing,  and 
simulator  sickness  following  each  of  the  sessions  are  shown  in 
Table  7.  Among  the  background  variables,  only  age  showed  a 
significant  correlation  pattern  with  the  task  variables,  with 
higher  age  associated  with  slower  task  performance.  The  mirror 
tracing  task  provided  a  better  predictor  of  VE  performance  than 
self  reported  weekly  averages  of  video  game  or  computer  use.  The 
data  also  hint  at  a  relationship  between  simulator  sickness  and 
task  performance,  with  those  participants  reporting  more  severe 
symptoms  performing  less  well  on  some  tasks. 


39 


Table  7 


Correlations  Among  VEPAB  Task  Time  Scores,  Age,  Mirror  Tracing, 
and  Simulator  Sickness  (SSQ)  Scores 


Age 

Mirror 

Simulator 

Sickness 

SSQ  Total 

SSQ  Total 

Tracing 

Severity  ~ 

Severity  - 

Day  1 

Day  2 

Locomotion  (walking) 


Straightaway 

.02 

(17) 

.41 

(15) 

.28 

(15) 

.42 

(12) 

Backup 

-.28 

(19) 

-.07 

(17) 

-.02 

(17) 

.01 

(14) 

Turns 

.51* 

(20) 

.58** 

(19) 

.52* 

(18) 

.39 

(16) 

Figure-8 

.42 

(20) 

.57** 

(18) 

.30 

(19) 

.09 

(16) 

Doorways 

.39 

(20) 

.59** 

(18) 

.44* 

(19) 

.26 

(16) 

1  Locomotion  (flying) 

Windows 

.62** 

(17) 

.70*** 

(17) 

.41 

(16) 

.34 

(15) 

Elevator 

.02 

(16) 

.01 

(16) 

.23 

(16) 

.10 

(15) 

Manipul a t i on 

Slide 

.56* 

(16) 

.51* 

(16) 

.49* 

(16) 

.42 

(15) 

Dial 

.15 

(16) 

.47* 

(16) 

.46 

(16) 

.36 

(15) 

Bins 

.50* 

(16) 

.47* 

(16) 

.46 

(16) 

.36 

(15) 

1  Tracking 

Head,  Stationary 

.05 

(16) 

-.21 

(16) 

-.19 

(16) 

.05 

(15) 

Head,  Moving 

-.14 

(15) 

-.33 

(15) 

-.24 

(15) 

-.25 

(14) 

Device,  Stationary 

.25 

(16) 

-.09 

(16) 

-.14 

(16) 

-.28 

(15) 

Device,  Moving 

-.01 

(15) 

-.27 

(15) 

-.08 

(15) 

.06 

(14) 

1  Reaction  Time 

Simple 

.00 

(16) 

-.03 

(16) 

.42 

(16) 

.40 

(15) 

Choice 

.34 

(16) 

-.06 

(15) 

.31 

(16) 

.20 

(15) 

Note:  Numbers  in  parenthesis  are  degrees  of  freedom. 
*E<.05  **£<.01  ***£<.001 
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Partial  Correlations  Amonq  Time  Scores  for  the  VEPAB 
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parenthesis  are  degrees  of  freedom 


Partial  Correlations  Among  Time  Scores  for  the  VEPAB  Tasks 


Choice _ I _ I _ I _ I _ 

STote :  Numbers  in  parenthesis  are  degrees  of  freedom 
*^£<.05  **£<.01  ***£<.001 


Partial  Correlations  Among 
Time  Scores  for  the  VEPAB  Tasks 


c 

tP 

>1 

•H 

C 

P 

X 

•H 

>1 

fd 

H 

>1 

P 

c 

(6 

H 

m 

o 

tP 

> 

IH 

c 

•H 

C  « 

>1 

a 

o 

tp 

■P 

*H  6 

0 

•H 

a 

fd 

>  H 

C 

12 

c 

•H 

4J 

•H 

-P 

O  H 

0 

OJ 

0 

(d 

> 

CO 

s 

•H 

i  CD 

•H 

M 

Id 

tP 

-P 

O 

fi 

1 

>t 

w 

O 

H 

C 

w 

s 

V  0 

g 

th 

a 

<u 

fd 

Q 

:s 

-P 

SJ 

•H 

<u 

0)  *H 

0)  0) 

-H 

p 

w 

M 

g 

o 

fd 

a< 

0) 

M 

o 

U  4J 

»— 1  o 

0 

rd 

c 

P 

M 

0 

TJ 

> 

■H 

TJ  r^ 

W 

U 

TJ 

TJ 

-H 

•H  0 

di  -H 

u 

o 

p 

O 

0 

C 

<D 

•H  (d 

C 

Id 

td 

td 

> 

>  Id 

g  o 

0 

-P 

fO 

0 

'H 

o 

0 

'H 

iH 

js 

H  -H 

-H 

M 

<U 

o 

0) 

0)  CJ 

•H 

m 

Eh 

Pm 

Q 

a 

IS 

pq 

2 

W  Q 

PQ 

HI 

Q 

Q  a: 

CO  U 

Numbers  in  parenthesis  are  degrees  of  freedom 


The  partial  correlation  coefficients  among  the  VEPAB  tasks, 
with  the  effects  of  control  device  removed,  are  presented  in 
Table  8.  Although  any  interpretations  must  be  limited  by  the 
small  niimber  of  participants  involved  (15  to  22,  depending  on  the 
particular  tasks) ,  some  patterns  appear  to  have  emerged. 
Locomotion  tasks  clustered  in  three  groups:  Straight-away  and 

Back-up;  Turns,  Figure-8,  and  Doorways;  and  Windows.  The  Elevator 
task  was  correlated  only  with  the  Turns  task.  The  Bins  task  was 
correlated  with  the  Slide  and  Dial  tasks,  which  were  not 
correlated  with  each  other.  Significant  correlations  were  found 
among  most  of  the  locomotion  and  manipulation  tasks.  The 
tracking  tasks  did  not  correlate  highly  with  other  tasks  or  among 
themselves.  Nothing  was  correlated  significantly  with  Simple 
Reaction  Time,  probably  reflecting  the  low  reliability  of  task 
performance.  Choice  reaction  time  was  correlated  with  a  few  of 
the  other  tasks. 

Side  effects  and  aftereffects 


Real-world  tests  of  vision  and  eye-hand  coordination  were 
presented  before  and  after  administration  of  the  VE  tasks  to  test 
for  aftereffects.  These  test  were:  Snellen  acuity,  contrast 
sensitivity,  stereopsis,  color  perception,  and  mirror- tracing. 

No  aftereffects  were  detected. 

Of  the  24  participants,  we  dismissed  one  from  the  experiment 
when  she  began  to  experience  nausea  during  the  locomotion  tasks . 

A  subsequent  interview  of  this  participant  revealed  a  history  of 
severe  susceptibility  to  motion  sickness.  The  other  participants 
stated  that  they  enjoyed  the  VE  experience  and  would  like  to 
participate  in  subsequent  VE  research. 

We  employed  the  Essex  Simulator  Sickness  Questionnaire  (SSQ) 
to  measure  simulator  sickness.  Our  participants  reported  total 
severity  scores  worse  than  those  of  the  Navy  aviators  surveyed  by 
Kennedy  et  al.  (1992).  Nevertheless,  with  the  exception  of  the 
one  participant  we  dismissed,  no  participant  reported  "severe" 
symptoms  of  any  kind.  Seven  of  the  participants  reported 
"moderate"  eyestrain  as  the  worst  symptom.  A  majority  reported 
"slight"  or  "none"  to  each  symptom.  The  partial  correlation 
coefficient,  with  Device  as  the  control  variable,  between  the 
total  severity  scores  for  the  first  and  second  VE  sessions  was 
.81,  p  <  .001,  indicating  that  participants  who  reported  higher 
incidence  of  symptoms  in  the  first  session  also  reported  higher 
incidence  in  the  second  session.  A  more  detailed  discussion  of 
our  findings  concerning  aftereffects  of  VE  immersion  is  presented 
in  Knerr  et  al.  (1993) . 
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Changes  to  VEPAB 


This  experiment  indicated  several  areas  in  which  VEPAB  could 
be  improved . 

Some  of  our  taller  participants  mentioned  that  their  estima¬ 
tion  of  the  height  of  objects  in  the  VEs  was  distorted  because  of 
the  discrepancy  between  their  real-world  eye  height  and  the  VE 
eye  height  which  was  arbitrarily  set  at  64  in.  for  all 
participants.  In  subsequent  research  we  will  set  the  VE  eye 
height  equal  to  that  of  each  participant's  real-world  eye  height. 

Some  of  the  locomotion  tasks  had  a  "finish  line"  which 
confused  the  participants.  This  has  been  removed.  Another 
problem  with  the  locomotion  tasks  involved  the  way  the  system 
counted  collisions.  A  collision  was  counted  for  each  frame  in 
which  the  simulated  body  was  in  contact  with  a  wall  or  door 
frame.  This  sometimes  resulted  in  dozens  of  collisions  being 
counted  for  what  seemed  to  the  participant  and  the  experimenter 
to  be  a  single  collision.  We  have  revised  the  rules  for  counting 
collisions  so  that  each  period  of  continuous  contact  is  counted 
as  one  collision,  regardless  of  its  duration.  However,  we  expect 
that  as  VE  hardware,  software,  and  peripherals  improve,  the 
incidence  of  collisions  for  even  novice  participants  will  decline 
to  the  extent  that  collisions  are  not  a  useful  measure  of 
performance. 

The  use  of  random  numbers  to  determine  the  initial  position 
and  direction  of  movement  of  targets  in  the  search  and,  to  a 
lesser  extent,  the  manipulation  and  tracking  tasks,  had  the 
intended  effect  of  preventing  participants  from  memorizing 
response  patterns,  but  led  to  differences  in  task  difficulty 
across  trials.  This  can  be  compensated  for  by  only  examining 
task  performance  on  groups  of  trial,  rather  than  single  trials. 

If  necessary,  a  limited  set  of  trials  of  comparable  difficulty 
can  be  developed  for  each  task. 

The  use  of  a  VE  model  of  a  Snellen  eye  chart  is  a  short-term 
approach  to  measuring  visual  acuity.  One  disadvantage  of  using 
the  Snellen  chart,  in  both  the  real  world  and  the  VE,  is  that 
participants  can  memorize  the  chart.  We  want  to  measure  acuity 
periodically  during  an  experimental  session  to  detect  changes 
that  may  occur  over  time.  (For  example,  acuity  may  improve 
through  perceptual  adaptation,  or  become  degraded  due  to  sensory 
fatigue  or  changes  in  hardware  resolution.)  We  attempted  to  do 
this  with  a  task  that  required  participants  to  discriminate  the 
orientation,  horizontal  or  vertical,  of  black  and  white  stripes. 
During  pilot  testing  we  found  that  the  discriminability  of  the 
stripes  fluctuated  non- systematically  as  a  function  of  viewing 
distance.  The  stripes  would  disappear  into  solid  fields  of  black 
or  white  and  then  become  discernable  again  at  greater  distances. 
An  implication  is  that,  in  contrast  to  real  world  acuity. 
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measuring  VE  acuity  at  a  given  distance  may  not  consistently 
allow  accurate  prediction  of  acuity  at  other  distances.  The 
development  of  acuity  measures  which  can  be  used  to  assess 
temporal  changes  in  acuity  and  factors  such  as  viewer  and  target 
movement  is  an  area  for  future  improvement. 

Finally,  as  a  longer-term  goal,  we  intend  to  add  audition 
tasks  to  the  battery.  Audition  tasks  will  involve  identification 
and  localization  of  sounds,  and  speech  intelligibility.  These 
tasks  will  be  developed  and  implemented  after  we  have  completed 
the  next  few  experiments  on  visual  display  devices. 

Conclusions 

This  first  experiment  reinforced  our  initial  subjective 
impressions  about  several  research  issues.  First,  the  data 
indicate  substantial  variability  in  initial  performance  among 
participants.  Some  participants  were  able  to  perform  the  tasks 
quite  well,  others  were  not.  One  implication  of  this  result  is 
than  future  research  (and  later,  VE  applications)  should  include 
provisions  for  training  participants  in  the  basic  VE  skills,  such 
as  locomotion  and  object  manipulation  that  are  required  for 
successful  task  performance.  The  VEPAB  itself  may  be  useful  in 
this  regard.  VEPAB  provided  a  systematic  orientation  for  naive 
participants  to  learn  how  to  perceive,  move  through,  and  interact 
with  objects  in  VEs. 

A  second  impression  that  was  reinforced  is  that  simulator 
sickness  is  a  potential  problem  for  the  use  of  VE  for  training. 
Although  only  one  participant  was  unable  to  complete  the 
experiment,  and  none  of  the  remaining  participants  reported  any 
severe  symptoms,  the  overall  level  of  discomfort  (mean  total 
severity)  was  higher  than  that  reported  by  naval  aviators 
following  training  sessions  in  flight  simulators.  While  naval 
aviators  might  differ  from  our  population  of  college  students  in 
several  ways  (less  susceptibility  to  motion  sickness,  and  less 
willingness  to  report  discomfort,  for  example) ,  our  results  do 
indicate  a  potential  problem,  and  provide  a  quantitative 
confirmation  of  the  anecdotal  reports  of  side  effects. 

This  research  has  established  a  set  of  tasks  that  can  be 
used  to  investigate  the  effects  of  interface  devices  on  human 
performance  and  learning.  In  general,  the  tasks  showed 
acceptable  reliability.  The  reliability  of  locomotion  accuracy 
scores  should  be  improved  by  the  changes  we  have  made  to  the 
method  used  to  count  collisions.  The  unreliability  of  the 
reaction  time  tasks  is  likely  due  to  the  slowness  of  the  PC-based 
systems  used  in  this  experiment.  Their  reliability  is  expected 
to  improve  when  the  tasks  are  performed  on  faster  systems. 

Although  tracking  of  moving  targets  was  poor,  all  of  the 
tasks  could  be  performed  by  our  diverse  group  of  participants. 
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who  varied  in  age,  sex,  and  familiarity  with  computers  and  video 
games.  Performance  on  most  tasks  was  sensitive  to  the  effects  of 
different  interface  devices  and  amount  of  practice.  This  shows 
them  to  be  free  of  artificial  limitations  on  performance  (e.g., 
ceiling  effects)  which  would  preclude  their  use  as  research 
tools.  While  five  of  the  tasks  showed  no  significant  practice 
effect,  we  expect  the  removal  of  the  apparent  barrier  at  the 
"finish  line"  of  the  Back-up,  Turns,  and  Windows  tasks  to  improve 
their  sensitivity.  Performance  on  the  HMD  Tracking  tasks,  on  the 
other  hand,  may  have  been  limited  by  system  parameters,  rather 
than  participant  skills. 

This  experiment  also  provided  baseline  data  on  human 
performance  of  a  variety  of  perceptual  and  motor  tasks  situated 
in  VEs. 

Our  research  to  date  with  VEPAB  has  reinforced  our  initial 
impressions  that  immersive  VE  technology  has  tremendous  potential 
to  support  a  wide  range  of  training  needs.  However,  we  believe 
that  to  realize  this  potential  efficiently,  systematic  research 
is  needed  on  technology  requirements  and,  as  important,  how  to 
use  the  technology.  The  availability  of  standard  VE  materials 
and  procedures  can  help  support  this  research.  VEPAB  is  a  step 
in  the  development  of  these  materials  and  procedures. 

The  contribution  of  the  VEPAB,  we  think,  lies  in  its  use  as 
a  tool  kit  of  tasks  which  can  be  used  to  design  and  evaluate 
proposed  VE  interfaces.  Tasks  can  be  selected  based  on  the 
research  needs  and  the  real-world  tasks  of  interest.  For 
example,  a  wide  variety  of  hardware  and  software  interfaces  for 
self -locomotion  in  VE  have  been  proposed  (and  implemented) .  How 
do  these  compare  in  terms  of  ease  of  use  for  novice  or 
experienced  users?  Do  they  differ  in  terms  of  the  feedback  they 
provide  to  the  user  about  distance  traveled  or  orientation  in  the 
VE?  How  does  system  update  rate  affect  self  locomotion?  Does 
"flying"  facilitate  or  hamper  locomotion?  A  subset  of  the 
locomotion  tasks  could  be  used  to  answer  these  and  similar 
questions.  Moreover,  the  answers  could  be  obtained  without  the 
expense  and  time  required  to  build  a  new  environment  for  testing. 
It  also  does  not  matter  whether  the  particular  application  is  to 
be  used  to  train  dismounted  soldiers  to  "clear"  buildings  or  home 
buyers  to  experience  their  soon  to  be  built  home,  the  results 
should  be  applicable  as  long  as  the  tasks  are  similar  to  those 
the  user  is  to  perform,  and  the  research  subjects  are 
representative  of  the  user  population. 

Developing  an  I -Port  for  the  dismounted  soldier  will  require 
a  large  number  of  interface  design  decisions.  Resources  to  build 
prototypes,  evaluate  them,  and  redesign  the  interface  are  likely 
to  be  limited.  The  VEPAB  provides  a  relatively  quick  and 
inexpensive  way  to  test  interface  devices  and  concepts. 
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Three  additional  experiments  with  VEPAB  are  scheduled. 

First,  we  will  use  a  subset  of  the  tasks  to  examine  the  effects 
of  extended  practice  on  performance,  and  the  extent  to  which  the 
control  device  effects  we  observed  in  this  experiment  persist.  We 
will  then  use  VEPAB  to  evaluate  performance  as  a  function  of  the 
characteristics  of  visual  displays.  One  experiment  will  compare 
three  displays:  a  standard  monitor,  the  Fake  Space  Lab  Binocular 
Omni -Orientation  Monitor  (BOOM) ,  and  the  Virtual  Research  Flight 
Helmet.  The  monitor  condition  provides  high  resolution  but  no 
head  tracking.  The  BOOM,  with  higher  resolution  than  the  Flight 
Helmet  and  faster  mechanical  tracking  than  our  Polhemus,  repre¬ 
sents  a  preview  of  improvements  we  expect  to  take  place  in  HMDs . 
Another  experiment  will  compare  task  performance  with  stereo¬ 
scopic  versus  monoscopic  HMDs,  with  and  without  head  tracking. 

To  support  these  latter  two  experiments  the  VEPAB  software 
has  already  been  modified  to  run  on  a  Silicon  Graphics  Crimson 
Reality  Engine.  The  Reality  Engine  will  produce  faster  update 
rates  than  the  PCs  used  in  our  first  experiment.  The  effects  of 
update  rate  on  performance  is  an  important  area  for  future 
research.  The  VEPAB  tasks  are  an  appropriate  tool  for 
investigating  those  effects  because  system  speed  is  not  an 
inherent  characteristic  of  the  VEPAB. 
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APPENDIX  A 


DISPLAY  SYSTEM  COMFORT  QUESTIONNAIRE 
(pre)  #  _ 

1.  Are  you  in  your  usual  state  of  fitness:  YES  NO 

If  not,  what  is  the  nature  of  your  illness  (flu,  cold,  etc) . 

2 .  Please  indicate  all  medication  you  have  used  in  the  past  24 
hours : 

(a)  NONE 

(b)  Sedatives  or  tranquilizers 

(c)  Aspirin,  Tylenol,  other  analgesics 

(d)  Anti-histamines 

(e)  Decongestants 

(f)  other  (specify) : 

3.  How  many  hours  sleep  did  you  get  last  night?  _  (Hours) 

Was  this  amount  sufficient?  YES  NO 
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Display  System  Comfort  Questionnaire'  (post) 


1. 

General  discomfort 

None 

Slight 

Moderate 

Severe 

2. 

Fatigue 

None 

Slight 

Moderate 

Severe 

3. 

Headache 

None 

Slight 

Moderate 

Severe 

4. 

Eye  Strain 

None 

Slight 

Moderate 

Severe 

5. 

Difficulty  focusing 

None 

Slight 

Moderate 

Severe 

6. 

Salivation  increased 

None 

Slight 

Moderate 

Severe 

7. 

Sweating 

None 

Slight 

Moderate 

Severe 

8. 

Nausea 

None 

Slight 

Moderate 

Severe 

9. 

Difficulty  concentrating 

None 

Slight 

Moderate 

Severe 

10. 

"Fullness  of  the  Head" 

No 

Yes 

11. 

Blurred  Vision 

No 

Yes 

12. 

a.  Dizziness  with  eyes  open 

No 

Yes 

b.  Dizziness  with  eyes  closed 

No 

Yes 

13  . 

Vertigo 

No 

Yes 

14. 

♦Stomach  awareness 

No 

Yes 

15. 

Burping 

No 

Yes 

No.  of  times 

16.  Other: 


*  Stomach  awareness  is  usually  used  to  indicate  a  feeling  of 
discomfort  which  is  just  short  of  nausea. 
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IMMERSION  QUESTIONNAIRES 

Indicate  your  preferred  answer  by  marking  an  "X"  in  the 
appropriate  box  of  the  seven  point  scale.  Please  try  to  use  the 
entire  scale  for  your  responses,  as  the  intermediate  levels  may 
apply.  For  example,  if  your  response  is  once  or  twice,  the 
second  box  from  the  left  should  be  marked.  If  your  response  is 
many  times  but  not  extremely  often,  then  the  sixth  (or  second  box 
from  the  right)  should  be  marked. 

1.  Do  you  ever  get  extremely  involved  in  projects  that  are 
assigned  to  you  by  your  boss  or  your  instructor,  to  the  exclusion 
of  other  tasks? 


NEVER  OCCASIONALLY  OFTEN 


2 .  How  easily  can  you  switch  your  attention  from  the  task  in 
which  you  are  currently  involved  to  a  new  task? 


NOT  SO  FAIRLY  QUITE 

EASILY  EASILY  EASILY 


3 .  How  good  are  you  at  blocking  out  external  distractions  when 
you  are  involved  in  something? 


NOT  VERY  SOMEWHAT  VERY  GOOD 

GOOD  GOOD 


4.  How  frequently  do  you  get  emotionally  involved  (angry,  sad, 
or  happy)  in  the  news  stories  that  you  read  or  hear? 


NEVER 


OCCASIONALLY 


OFTEN 


5.  Do  you  easily  become  deeply  involved  in  movies  or  tv  dramas? 


NEVER 


OCCASIONALLY 


OFTEN 
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6 .  Do  you  ever  become  so  involved  in  a  television  program  or 
book  that  people  have  problems  getting  your  attention? 


NEVER  OCCASIONALLY  OFTEN 

7.  Do  you  ever  become  so  involved  in  a  movie  that  you  are  not 
aware  of  things  happening  around  you? 


NEVER  OCCASIONALLY  OFTEN 

8.  How  frequently  do  you  find  yourself  closely  identifying  with 
the  characters  in  a  story  line? 


NEVER  OCCASIONALLY  OFTEN 


9 .  Do  you  ever  become  so  involved  in  a  video  game  that  it  is  as 
if  you  are  inside  the  game  rather  than  moving  a  joystick  and 
watching  the  screen? 


NEVER  OCCASIONALLY  OFTEN 

10.  On  average,  how  many  books  do  you  read  for  enjoyment  in  a 
month? 


NONE  ONE  TWO  THREE  FOUR  FIVE  MORE 


11.  What  kind  of  books  do  you  enjoy  most  and  read  most 
frequently?  (CIRCLE  ONE  ITEM  ONLY!) 


Adventure  novels 

Westerns 

Fantasy 


Autobiographies 
Romance  novels 
Mysteries 


Science  fiction 
Historical  novels 
Spy  novels 


12.  How  physically  fit  do  you  feel  today? 


NOT  FIT  MODERATELY 

FIT 


EXTREMELY 

FIT 
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13. 


How  well  do  you  feel  today? 


NOT  WELL 


PRETTY  EXCELLENT 

WELL 


14.  How  mentally  alert  do  you  feel  at  the  present  time? 

I _ I _ I _ I _ I _ I _ ^ _ I _ I 

NOT  ALERT  MODERATELY  FULLY  ALERT 

15.  To  what  extent  have  you  dwelled  on  personal  problems  in  the 
last  48  hours? 

I _ _l _ I _ I _ I _ I _ I _ I 

NOT  AT  ALL  SOME  ENTIRELY 

16.  When  watching  sports,  do  you  ever  become  so  involved  in  the 
game  that  you  react  as  if  you  were  one  of  the  players? 

I _ I _ I _ I _ I _ I _ I _ I 

NEVER  OCCASIONALLY  OFTEN 

17.  Do  you  ever  become  so  involved  in  a  daydream  that  you  are 
not  aware  of  things  happening  around  you? 


NEVER  OCCASIONALLY  OFTEN 

18.  Do  you  ever  have  dreams  that  are  so  real  that  you  feel 
disoriented  when  you  awake? 


NEVER 

OCCASIONALLY 

OFTEN 

19.  When  playing  sports,  do  you  become  so  involved  in 
that  you  lose  track  of  time? 

1  1  1  1  1  1  1 

the  game 

1 

NEVER 

OCCASIONALLY 

OFTEN 

2  0 .  Are 

1 

you  easily  disturbed  when  working  on  a  task? 

_L .  1  1.  1  . .  1  1 

1 

NEVER 

OCCASIONALLY 

OFTEN 
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21. 


How  well  do  you  concentrate  on  enjoyable  activities? 


NOT  AT  ALL 


MODERATELY 

WELL 


VERY  WELL 


22.  How  often  do  you  play  arcade  or  video  games?  (OFTEN  should 
be  taken  to  mean  every  day  or  every  two  days,  on  average.) 


NEVER  OCCASIONALLY 

23.  How  well  do  you  concentrate  on  disagreeable  tasks? 

I _ 1 _ I _ I _ I _ I _ I 

NOT  AT  ALL  MODERATELY 

WELL 


OFTEN 


VERY  WELL 


24.  Have  you  ever  gotten  excited  during  a  chase  or  fight  scene 
on  TV  or  in  the  movies? 


NEVER 


OCCASIONALLY 


OFTEN 


25.  Have  you  ever  gotten  scared  by  something  happening  on  a  TV 
show  or  in  a  movie? 


NEVER  OCCASIONALLY  OFTEN 

26.  Have  you  ever  remained  apprehensive  or  fearful  long  after 
watching  a  scary  movie? 


NEVER  OCCASIONALLY  OFTEN 

27.  Do  you  ever  avoid  carnival  or  fairground  rides  because  they 
are  too  scary? 


NEVER 


OCCASIONALLY 


OFTEN 
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28.  How  frequently  do  you  watch  tv  soap  operas  or  docu-dramas? 


NEVER 


OCCASIONALLY 

PLEASE  DO  NOT  PROCEED  UNTIL  REQUESTED! 
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OFTEN 


NOT  AT  ALL  SOMEWHAT  COMPLETELY 


6 .  How  much  did  the  auditory  aspects  of  the  environment  involve 
you? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


7.  How  natural  was  the  mechanism  which  controlled  movement  through 
the  environment? 


EXTREMELY  BORDERLINE 

ARTIFICIAL 


COMPLETELY 

NATURAL 


8.  How  aware  were  you  of  events  occurring  in  the  real  world  around 
you? 


NOT  AWARE  MILDLY*  VERY  AWARE 

AT  ALL  AWARE 


9.  How  aware  were  you  of  your  display  and  control  devices? 


NOT  AWARE  MILDLY*  VERY  AWARE 

AT  ALL  AWARE 


10.  How  compelling  was  your  sense  of  objects  moving  through  space? 


NOT  AT  ALL 


MODERATELY  VERY 

COMPELLING  COMPELLING 


11.  To  what  degree  did  you  experience  disconnects  or 
inconsistencies  between  the  information  coming  from  your  various 
senses? 


NO  INCON-  SOME  INCON-  MANY  INCON¬ 
SISTENCIES  SISTENCIES  SISTENCIES 


12 .  To  what  degree  did  your  experiences  in  the  virtual  environment 
seem  consistent  with  your  real  world  experiences? 


NOT  MODERATELY  VERY 

CONSISTENT  CONSISTENT  CONSISTENT 
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13 .  To  what  degree  were  you  able  to  anticipate  what  would  happen 
next  in  response  to  the  actions  that  you  performed? 


NOT  AT  ALL 

SOMEWHAT 

COMPLETELY 

14 .  How  completely  were  you  able  to  actively  survey 
environment  using  vision? 

1  1  1  1  1  1 

or  search  the 

1  1 

NOT  AT  ALL 

SOMEWHAT 

COMPLETELY 

15 .  How  well  could  you 

1  1  1 

identify  sounds? 

1  1  1 

1  1 

NOT  AT  ALL 

SOMEWHAT 

COMPLETELY 

16.  How  well  could  you 

1  1  1 

localize  sounds? 

. . 1 . .  .  1  1 

. r.  1 

NOT  AT  ALL 

SOMEWHAT 

COMPLETELY 

17.  To  what  extent  were  you  able  to  actively  survey  or  search  the 
virtual  environment  using  touch? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


18 .  How  compelling  was  your  sense  of  moving  around  inside  the 
virtual  environment? 


NOT  MODERATELY  VERY 

COMPELLING  COMPELLING  COMPELLING 


19.  How  closely  were  you  able  to  examine  objects? 


NOT  AT  ALL 


PRETTY 

CLOSELY 


VERY 

CLOSELY 


20.  Were  you  able  to  examine  objects  from  multiple  viewpoints? 


NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 
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21.  Were  you  able  to  move  or  manipulate  objects  in  the  virtual 
environment? 


NOT  AT  ALL  SOMEWHAT  EXTENSIVELY 


22 .  To  what  degree  did  you  feel  confused  or  disoriented  at  the 
beginning  of  breaks  or  at  the  end  of  the  experimental  session? 


NOT  AT  ALL 

MILDLY 

VERY 

DISORIENTED 

DISORIENTED 

23.  How  involved  were  you  in  the  virtual  environment  experience? 

1  1  1  1  1  1  1  1 

NOT 

MILDLY 

COMPLETELY 

INVOLVED 

INVOLVED 

ENGROSSED 

24 .  Was  the  control 

1  i  1 

mechanism  distracting? 

.  1  L__ . .  1 

1  _  i 

NOT  AT  ALL 

MILDLY 

VERY 

DISTRACTING 

DISTRACTING 

25.  How  much  delay 
expected  outcomes? 

1  1  1 

did  you  experience  between 

1  1  1 

your  actions  and 

1  1 

NO  DELAYS 

MODERATE 

LONG 

DELAYS 

DELAYS 

26.  How  quickly  did  you  adjust  to  the  virtual  environment 
experience? 


NOT  AT  ALL  SLOWLY  IN  LESS 

THAN  ONE 
MINUTE 


27.  How  proficient  in  moving  and  interacting  with  the  virtual 
environment  did  you  feel  at  the  end  of  the  experience? 


NOT  REASONABLY  VERY 

PROFICIENT  PROFICIENT  PROFICIENT 
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28.  To  what  extent  did  the  visual  display  quality  interfere  or 
distract  you  from  performing  assigned  tasks  or  required  activities? 


NOT  AT  ALL  INTERFERED 


SOMEWHAT 


PREVENTED 
TASK  PER¬ 
FORMANCE 


29.  To  what  extent  did  the  control  devices  interfere  with  the 
performance  of  assigned  tasks  or  with  other  activities? 


NOT  AT  ALL 


INTERFERED 

SOMEWHAT 


INTERFERED 

GREATLY 


30.  How  well  could  you  concentrate  on  the  assigned  tasks  or 
required  activities  rather  than  on  the  mechanisms  used  to  perform 
those  tasks  or  activities? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 


31.  Did  you  learn  new  techniques  that  enabled  you  to  improve  your 
performance? 


NO  LEARNED 

TECHNIQUES  SOME 

LEARNED  TECHNIQUES 


LEARNED 

MANY 

TECHNIQUES 


32 .  Were  you  involved  in  the  experimental  task  to  the  extent  that 
you  lost  track  of  time? 


NOT  AT  ALL 


SOMEWHAT 


COMPLETELY 
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